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Introduction 

Programming  may  be  defined  as  the  construction  of  a  schedule 
of  actions  by  means  of*  which  an  organization  or  complex  of  activ- 
ities may  move  from  a  defined  state  toward  some  specifically  de- 
fined objective.   Such  a  schedule  should  explicitly  prescribe  the 
resources  and  the  goods  and  services  utilized,  consumed,  or  pro- 
duced in  the  accomplishment  of  the  programmed  actions.   The  re- 
sources and  the  goods  and  services  utilized,  consumed,  or  pro- 
duced by  the  activities  may  be  referred  to  as  "commodities". 

Linear  Programming  is  the  search  for  a  program  which  will, 
in  some  sense,  most  nearly  accomplish  the  desired  objectives 
without  exceeding  stated  resource  limitations.   To  accomplish 
this,  all  interrelationships  in  the  organization  or  complex  of 
activities  must  be  represented  by  a  system  of  simultaneous  linear 
relations  in  which  the  variables  are  the  quantities  of  the  activ- 
ities to  be  performed,  the  coefficients  are  the  requirements  of 
each  activity  for  each  commodity,  and  each  linear  relation  ex- 
presses the  relationship  between  the  sum  of  the  requirements  of 
all  activities  for  a  single  commodity  and  the  outputs  of  that 
commodity  from  all  activities.   In  preparing  such  a  program,  it 
is  necessary  to  insert  in  the  system  a  specification  of  the  ini- 
tial status  in  terms  of  the  quantities  of  each  commodity  on  hand, 
any  subsequent  limitations,  and  a  statement  of  the  objectives. 
Thus,  the  determination  of  the  desired  program  mathematically  in- 
volves the  solution  of  a  system  of  linear  relations,  and  hence, 
is  called  Linear  Programming. 


It  is  generally  possible  to  determine  the  linear  program  us- 
.  irig  a  mathematical  model  which  will  maximize  the  accomplishment 
of  given  objectives  within  those  stated  resource  limitations. 
Alternately,  it  will  be  possible  to  determine  the  program  which 
will  minimize  the  requirements  for  any  group  of  commodities  need- 
ed to  accomplish  any  fixed  objective. 

The  origin  and  background  of  linear  programming  has  been 
primarily  in  the  field  of  econometrics  and  research  into  economic 
relationships.   The  history  of  linear  programming  dates  from  the 
publication  of  work  of  the  methematical  economist  Leon  W'alras  in 
1874-".   V/alras  showed  that  the  price  .of  any  number  of  commodities 
at  a  single  time  can  be  determined  by  solving  simultaneously  the 
correct  number  of  equations  in  terms  of  the  number  of  unknowns 
for  which  a  solution  is  sought.   At  that  time,  the  concept  was 
revolutionary,  but  present-day  methods  of  linear  programming  are 
completely  different  from  those  used  by  w'alras.   However,  it  was 
this  first  attempt  to  solve  programming  problems  by  stating  the 
problem  conditions  in  equation  form  that  provides  the  connection 
between  itfalras  and  linear  programming. 

Linear  programming,  as  it  is  known  today,  began  with  the  in- 
put-output method  of  analysis  developed  by  the  economist  Wassily 
■V.  Leontief  in  the  1920's.   The  present-day  development  stems 
primarily  from  the  work  of  George  B.  Dantzig.   Dantzig  is  credited 
with  developing  the  Simplex  Method  of  linear  programming,  which 
is  essentially  a  method  of  solving  simultaneous  equations  and  in- 
equalities Tor  an  optimum  or  best  solution  (a  proof  of  the  Simplex 
Method  is  given  in  the  Appendix).   Since  Dantzig  announced  his 


development  in  194-7 >  the  adaptation  and  application  of  the  simplex 
method  to  solving  linear  programming  problems  has  been  fostered 
by  simultaneous  computational  advances  through  high-speed  digital 
co mpu t i ng  ma chines. 

It  is  to  be  emphasized  that  the  simplex  method  is  not  the 
only  linear-programming  method.   However,  most  known  methods  are 
derived  or  have  evolved  from  the  simplex  mechod,  which  is  consid- 
ered to  be  the  fundamental  linear-programming  method.   Some  of 
these  methods  are:   the  Index,  the  Modi,  the  Ratio  Analysis,  and 
the  Symmetric  Methods. 

The  aim  of  this  report  is  to  give  a  presentation  of  the 
theory  of  linear  programming  utilizing  the  concept  of  duality 
with  respect  to  the  maximization  and  minimization  problems.   A 
purely  algebraic  approach  has  been  employed.   Whenever  possible, 
the  more  convenient  matrix  notation  has  been  used  to  represent 
systems  of  linear  relations.   No  attempt  has  been  made  to  give  a 
complete  geometric  interpretation  of  any  result. 
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An  interpretation  of  the  maximization  problem  may  be  given 
iii  terms  of  Activity  Analysis  of  Production.   Let  there  be  n 
activities,  i.e.,  ways  of  making  a  single  desired  commodity  from 
available  stocks  of  m  primary  materials.   Let  a .  .  be  the  amount 

J 

of  the  i-th  material  used  in  one  unit  of  the  ,j-th  activity,  b. 

the  available  stock  of  the  i-th  material,  c.  the  quantity  of  the 

J 

desired  commodity  made  by  one  unity  of  the  j-th  activity,  and  x. 

J 
the  number  of  units  of  the  j-th  activity  to  be  undertaken.   The 

maximization  problem  is  then  a  search  for  an  activity  vector 
(*]_  ,  x2  ,  . . .  ,  xn)  which  will  yield  the  greatest  possible  out- 
put (1.1)  of  the  desired  commodity,  subject  to  the  constraints 
(1.2)  set  by  available  stocks  of  the  m  materials  and  by  the  natu- 
ral impossibility  (1.3)  of  negative  activity  levels. 

The  dual  problem  pertains  to  accounting  (fictitious  or  shad- 
ow) prices  attached  to  the  m  materials,  on  a  scale  whose  unit  is 
the  price  of  the  desired  commodity.   In  solving  this  problem,  one 
seeks  a  price  vector  (ux,  u2,  ...  ,  um)  that  minimizes  the  total 
accounting  value  (1.4)  of  the  available  stocks  of  materials,  sub- 
ject to  the  requirement  (1.5)  that  the  accounting  values  of  the 
quantity  of  the  desired  commodity  made  by  one  unit  of  an  activity 
can  never   exceed  the  total  accounting  value  of  the  materials  used 
in  that  unit,  and  to  the  natural  requirement  (1.6)  that  all 
accounting  prices  be  nonnegative.  . 

Returning  to  the  mathematical  discussion,  the  dual  linear 
programming  problems  will  now  be  stated  in  the  more  compact  matrix 


notation  to  be  employed  in  the  development  of  the  theory  in  this 
report.   All  vectors  will  be  from  an  n-dimensional  real  vector 
space  vn  and  will  be  denoted  by  upper  case  letters.   The  compo- 
nents of   a  vector  will  be  denoted  by  lower  case  letters.   The 
transpose  of  a  matrix  X  will  be  denoted  by  XT.   In  particular, 
let  U  =   (\i-j_  ,  u2  ,  ...  ,  um)  and  C  =  (C;L  ,  c2  ,  ...  ,  c  )  be  row 
vectors  and  let  X  =  (x-,  ,  x2  ,  .  .  .  ,  xn)  and  a   =  (t^  ,  t>2  ,  . . .  , 
bm)  be  column  vectors.   However,  vectors  U,  B,  and  C  will  be  used 
interchangeably  as  row  and  column  vectors.   The  use  of  inequality 
signs  in  terms  of  vectors  will  be  as  follows: 

X>Y  means  x±>  ;/±  for  all  i, 
X^Y  means  x.^  y .  for  all  i, 
XSY   means  x^  7.±      for  all  i,   :<i>  y±   for  some  i. 

The  basic  problems  for  initial  discussion  then  can  be  written  in 
canonical  form  using  matrix  notation  as: 

(1*7)     I  Maximize  CX,  subject  to  AXiB,   X50. 
(1.8)    II  Minimize   UB,  subject  to   UA^C,   U^O. 

A  vector  X  satisfying  the  m  +  n  constraints  (1.2)  and  (1.3) 
without  necessarily  yielding  the  maximum  in  (1.1)  will  be  called 
a  feasible  vector  or  solution  for  the  maximization  problem.   A 
feasible  vector  X  which  provides  the  desired  maximum  for  CX  will 
be  called  an  optimal  vector  or  solution  for  the  maximization 
problem. 

The  terms  feasible  solution  and  optimal  solution  are  defined 


analogously  for  the  minimization  problem. 

Lemma  1.1.   If  X  and  U  are  feasible,  then  CX^UB. 

Proof:   Consider  the  constraints  of  the  problems  (1.7)  and  (1.8). 
Upon  substituting  and  re-associating,  it  follows  that 
CXt=(UA)X  =  U(AX)^UE. 

Lemma  1.2.   If  X°  and   U°  are  feasible,  and  CX°  =  U°B, 
then  X  and  U°  are  optimal. 

Proof:   If  CX°  =  U°B,  then  by  Lemma  1.1,   U°B  =  CX°£  UB  for 
all  feasible  vectors  U.   Thus  U°  is  optimal.   If  CX°  =  U°B, 
by  Lemma  1.1,   CX  g  U°B  =  CX°   for  all  feasible  vectors  X.   Thus 
X  is  optimal. 

According  to  Lemma  1.2,  the  assertion  that  X°  and  U°  are 
a  pair  of  optimal  vectors  can  be  checked  directly  in  the  schematic 
form  of  the  dual  programs: 

1.  One  checks  the  feasibility  of  X°  by  making  sure  that 
all  its  components  are  nonnegative  and  that  the  inner  product  of 
each  row  of  the  matrix  A  with  X°  is  not  greater  than  the 
corresponding  b.. 

2.  One  checks  the  feasibility  of  U°  by  making  sure  that 
al3  its  components  are  nonegative  and  that  its  inner  product  with 
each  column  of  AT  with  U°  is  not  less  than  the  corresponding 


V 


3.   Finally,  one  checks  the  equality  of  the  inner  product 


of   _  and  X°  with  that  of  U°  and  B 
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Lemma  1.3.   If  B|0,  then  the  maximization 

problem  has  fea- 

sible  constraints.   I 

et  the  n  columns  of  the  matrix  A  be 

denoted  by  A,,..., A  . 

Then,  if  A.S  0  for  each 

i,  the  minimiza- 

tion  problem  has  feasible  constraints. 

Proof:   The  validity 

of  the  first  statement  is  seen  by  noting 

that  the  zero  vector 

is  feasible  for  the  maximization  problem. 

The  second  statement 

is  proved  by  noting  that  under  the  hypothe- 

sis,  any  vector  U  will  be  feasible  if  all  its 

•components  are 

sufficiently  large . 

The  fallowing  two  examples  illustrate  the  possibility  that 

constraints  in  one  or 

both  of  the  dual ' problems 

may  not  be 

feasible. 

Example  1. 

xl  "  5x2~2 

u1  +  4uP"g3 

4oc,  -  2x^-^0 

-3ux  -  2u2^l 

xx§  0,   x9^0. 

ul=°>   u2~ 

0. 

Here,   X  =  (1,2)T  is 

feasible  but  the  column  constraints  clearly 

cannot  l>«  met,  so  the 

minimization  has  no  feasible  solution 

vector. 

Example  2. 

0x1  +  Xp^-1 

Ou-j^  +  0u2^  1 

0x1  +  4x2<  3 

u1  +  4u2"5  2 

x,£  0,   xo5-0. 

u1S  0,   u2^ 

0. 

Here,  neither  the  row  constraints  or  the  column  constraints  can 
be  satisfied,  so  neither  problem  has  a  feasible  solution  vector, 
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Dual  Systems  of  Homogeneous  Linear  Relations 

In  this  section,  theorems  concerning  dual  systems  of  homo- 
neous  linear  relations  are  derived.   These  theorems  play  an 
important  role  in  the  development  of  duality  and  existence 
theorems  of  the  next  section.   Throughout  this  section,  the 
letters  A,  3,  C,  and  D  will  denote  m  x  n  matrices  with  arbitrary 
real  elements.   Also,  the  letters  U  and  V  will  denote 
m-dimensional  vectors  from  the  vector  space   V   while  the 

letters  X  and  Y  will  denote  n-dimensional  vectors  from  V  . 

T 
First,  consider  the  pair  of  dual  systems  A  U^O  and 

AX=0,   X^O  where   A  =  (A1, A2,  .  .  .  »A  )       is  an  n-columned 

matrix  with  arbitrary  real  elements. 

Lemma  2.1.   The  dual  systems  A  U2T0  and  aX=0,   X^O 

T 
possess  solutions  U  and  X  such  that  A.U  +  x.>0,  where 

A.   is  any  one  of  the  n  columns  of  A. 

Proof:  The  proof  is  by  induction  on  n,  the  number  of  columns 
in  A.  The  initial  case,  n  ■  1,  is  trivial:  If  A.  =  0,  take 
U  =  0  and  x±      =  1:  if  A^O,  take   U  =  A.  and  x.  =  0. 

wow  assuming  that  the  Lemma  holds  for  a  matrix  A  of  n 
columns,  it  will  be  shown  that  the  Lemma  holds  for  a  matrix 
A  =  (A»An+]_)  =  (A^, . . .  »An»An+i)   of  n  +  1  columns.   Applying 
the  Lemma  to  A,  one  obtains  vectors  U  and  X   such  that 
A1U10,   AX  =  0,  X5  0,  and   aTu  +  x.>0. 

If  An+lU-°>  take  7-   =  (X>°)T-   ^en  aTU£0,   3X  =  0, 

-  m 

X  =0,   AiTJ  +  x.^>  0,  which  extends  the  Lemma  to  the  matrix  X. 
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.  T 
However,  if  An  1U<0,  an  application  of  Lemma  2.1  to  a 

r  n  x  ra  matrix  3  =  (B^Bg, .  .>  ,B  )   will  again  extend 
the  Lemi  a  to  the  matrix  A. 

Lct  B  =  <B1>  —  »V  '-  (A1  +  klAntl-'-'An  +  knVl>> 

where  the  constants  k.   are  given  by  k.  =  -A~U/A™  ,U 

J  J     J    n+1 

for  j  =  l,...,n.  Note  that  the  k.  are  nonnegative  since 

m  J 

A  UfcO.   This  application  of  the  Lemma  to  the  matrix  B 
yields  vectors   V  and  Y  such  that  BTV£0,   3Y  =  0,   YfcO, 

m 

and  B±V  +  y±>  0.   Then,  if   i   is  chosen  to  be 


~  =  (Y'  SrVd^'  U   follov;s  ^0,  since  Y  sL  0  and  ilk.y.^O. 


Furthermore,  AY  =  BY  =  0.   This  can  be  seen  by  writing 

I?  -  (  g  *l/j  +  %n*l  ^yd--  £an/j  +  am,n.l  £  *// 

-  BY, 


and  by  noting  that   BY  =  0  by  hypothesis. 

Now,  let  W  =  V  ■+  rU  ,  so  that  if  r  is  chosen  to  be 
~An+lV/An+lU>   then  A  W»Q.   The  latter  relation  follows  by 


»  •'<  ■  (AjW A^V.')1 

■  (a^v  -  &£  aTu,...,a;v  -  ClT  A)I 
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rn*l 

«T  „     ,T„   ATU 
An+lV'""AnV  "  § 

VlU 

n-rl  ' 

rn          m             m           mm 

=  (A(V  +  k,Ar  nV,.'  A„V  +  k  A*  ,  v)1 

1     1  n+1  '    '  n     n  n-*-l  ' 

=  BTV 2  0 , 

and  by  noting  that 

_           _            T>                          m 

T          m           '  x    \T                          m           J.    -57- 

An+1W  -  A;t1(V  -  W  U)  =  A'  +  1V  -  ^1V 
An.lU             VlU 

.T 

-n,lU  "  °« 

Therefore,  ATW 

+  yi  =  BiV  +  vi>0  for  i  =  l,...,n. 

Thus;  by  means 

of  £he  vectors  W 

Y,  the  Lemma  has 

been  extended  to  the 

matrix  S  =  (A, A  , ) . 
'  n+ly 

Therefore ,  the 

indueti on  Ox.  n  is 

fully  established  and 

the  Lemma  must  hold 

for  all  n. 

The  following  corollary  is  a  statemen 

t  of  a  theorem  first 

roved  in  1902  by  J. 

Parkas  (9). 

Corollary  2.1. 

m 

If  the  inequality  A 

JSO  holds  for  all 

solutions   U  of  the 

system  A  UiO,  then 

AQ   =  AX  for  some 

XfcO. 

Proof:   Consider  the 

T 
matrix   (-AQ,A)   whose  first  column 

is  -A  ,  and  the  dual 

systems  (-A0,A)%8  0 

and 

C-A0,A)  CxoiX)T«0> 

m 

(x  ,X)  £0.'  Note  that  -A„  and  x 
w                                                o        o 

assume  the  roles  of 

A.  and  x.   with  i  = 

1  in  Lemma  2.1. 
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By  bhi;     ..a,  one  is  assured  of  some  solution  U  and 

(xQ,X)T  such  that   -iV?U£0,   ATU£0,   -Ax  +  AX  =  0, 

x^^O,   X^'S  0,  and  -A^U.+  x  >  0.   The  relation  -A^U  +  x  >0 
0        0'         o      o  o      o 

implies  that  x  >  0,  since  by  hypothesis  of  the  Corollary, 

..:U*0  for  all  solutions,   U  of  the  system.   Thus,  from  the 

relations  -A  x^  +   AX  =  0  and  x  >  0,   it  follows  that 
o  o  o 

=  kX    ,   where  X  =  X/x  ^  0. 

It  is  to  be  emphasized  that  the  vector  A   in  the  above 

o 

corollary  is  actually  a  nonnegative  linear  combination  of  the 
vectors   A, , . . . , A  ,   that  is, 

AQ  =  AlXl/x0  +  A2x2/xQ  i-\.\   Anxn/xQ, 

-  Al7l   +  A2y2  +...  +  Anyn, 

with  y^O,   for  i  =  l,...,n. 

With  this  remark,  the  corollary  may  be  interpreted  as 
follows:   if  a  linear  inequality  is  dependent  upon  a  system  of 
linear  inequalities,  in  the  sense  that  it  is  satisfied  by  all 
solutions  of  the  system,  then  the  coefficients  of  -chat  inequality 
depend  linearly  on  the  coefficients  of  the  system,  and  further- 
more, in  this  dependence  only  nonnegative  coefficients  need  be 
used. 

The  following  theorem  is  a  key  result  because  from  it 
virtually  all  of  the  results  in  duality  theory  follow. 

Theorem  2.1.   The  dual  systems   ATU  5  0   and  AX  =  0,  X§0 
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*        *  <p  *     * 

possess  solutions   U   and   X   such  that  A*U  +  X  >  0. 

Proof:   By  Lemma  2.1,,  there  exist  pairs  of  solutions  U^ 
and  XJ   such  bhat  aV  S  0,   AX^=0,   X^SO,  and 
A^UJ  +  xJ;>0.   Let  \J*=i£vJ      and   X*=  it   X J' .   Then, 

sZ2ATuJ^0,   AX**SAX^»0,   and  X*=  ^T  X^G. 

«  j  j 

io,  for  j  =  1,. ..  ,n, 


n   *        *      H      mi.  i        m   -;        -: 

A^U   +  x  =  2Z  (A:Uk  +  x*)  S  ATUJ  +  x^  >  0, 

since  A*U  >0  and  X&iO  for  k  =  l,...,n.   Therefore, 
AXU  +  X  >  0. 


It  is  interesting  to  note  that  the  solutions  U   and  X 
of  Theorem  2.1  have  the  property  that  for  J  =  l,...,n,  the 

j-th  component  of  one  of  the  vectors  X*  and  ATU*   is  zero  if, 

and  only  if,  the  j-th  component  of  the  other  is  positive. 

Corollary  2.2.   The  system  of  equations  AX = 0  has   (i) 

a  solution  X>0  if  there  is  no  solution  U  such  that 

T 
A  U>0,   and  (ii)  a  solution  X>  0  if  there  is  no  solution 

U  such  that  ATU >  0. 

Proof:   By  Theorem  2.1,  there  exist  solutions  U*   and  X* 

T  *  *       *  m  *      * 

suen   that      A   U    £0,      AX  =0,    X    £  0,    and     Al\]     +   X    >  0. 

From  the  hypothesis  of  Theorem  2.1,  every  solution  U  must 

satisfy  Lhe  relation  A  UgO.   Thus,  if  it  is  not  true  that 

T 
A  UiO  for  every  solution  U,   then  the  mutually  exclusive 


alternative  A  U  =  0  must  be  true.   In  particular,   A  J   =  0 

T  *  * 

if  A  U  >  0  does  not  hold.   Gince  U   also  satisfies 
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AXU 

+  X  >  0,   it  follows  bhat  > 

:*>0.   This 

establishes  part 

(i). 

Now,  if  any  solution  X  of 

the  system 

AX  -  0, 

X£0 

does 

.  t  satisfy  the  relation  X> 

0,   then  X 

must  be 

zero. 

* 
the  particular  solution  X  , 

this  implies 

,  that  A 

rp   * 

XU  >  0, 

m  *       * 

since  A  U  +  X  >  0.   The  contrapc 

sitive  of  tl" 

is  result 

esta 

blishes  part  (ii),  and  hence, 

completes  t'r 

e  proof. 

Theorem  2.2.   The  dual  systei: 

is  ATU£0, 

BXU  =  0 

and 

....   + 

BY  =  0,  XgO  possess  solutions  U   and 

*    * 
X  ,  Y 

such  that 

* 

+  X   >  0. 

Proof:   The  application  of  the  res 

ults  of  Theorem  2.1 

to  the 

dual 

systems   (A,B,-B)  Ufc  0  and- 

(A,B,-B)(X,Y 

v  )T  _ 

o, 

(  '■'  Y 

T                               '               * 
pY^)  fc  0  implies  the  existence  of  solutions   U 

and 

* 
-•  » 

Y-,  ,  Yp   such  that  ■ 

,.-'J  £  0,   B^U  >  0,   -BUT  £  0, 

*     * 
+  BY,- 

-3Yp=0, 

and 

X***  0,   Y^O,   Yp^O, 

and 

•T  *       *           m  *       * 

AAU  +  X  >  0,  BXU  +  y,>  0, 

-BXU   +  Yp>  0. 

If  o. 

*       *      * 

te  dhooses  Y  =  Y,  -  Yp,   then 

AXU  >   0,   BXU  -  0,   AX   +  BY 

,  clear!;.', 
=  0,   X*3E  0, 

m  *      * 

AXU   +  X  >  0. 

Corollary  2.3.   Let  the  dual 

ATU£0,   BTU  =  0   and  AX  +  BY  = 

systems 
■o,  XfcO 

have 

the  partitioned  representation 
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IT 
Li  j 

0,   A2T£0,   BTU=0   and  A1X1  +  A1X2  +  BY=0, 

Xj*  o, 

XP$  0,   where   A   is  any  nonvacuous  set  of  columns 

of   A   and 

A   is  the  set  (possibly  vacuous)  of  remaining  columns  of  A. 

Then 

(i)  either  the  left  system  has  a  solution  U   a 

,uch  that 

IT 
AX1U 

^0  or  the  right  system  has  a  solution  X  suck 

t  that 

X,  >  0.   Also,   (ii)   either  the  left;  system  has  a  sc 

ilution  U 

such 

IT 
that   A  XU>0  or  the  right  system  has  a  solut 

;ion  X 

such 

that  X,  i  0. 

* 
Proof:   Theorem  2.2  asserts  the  existence  of  solutions  U 

and 

*    *   * 
X,,  X0,  Y    such  that 

AXiU  5  0,   A^1U  fcQ,  BXU  -  0,  A   X±   +   A^X2  +  BY 

=  0, 

and 

X^g'O,   X-^O,   A±1U  +  X,>0,   A  ^U  +  X2>0. 

?or  i 

IT 

iny  solution  U  to  satisfy  the  relation  A  U  = 

0,  one 

1  rp 

of  the  mutually  exclusive  alternatives,   A  ~ =  0  or 

1  T 
A11  2  0, 

must 

*    11 
hold.   If  for  the  particular  solution  U  ,   A 

U  ■  0, 

then 

X,  >  0  ,  since   A  1tJ  +  X,  >0.   This  establishe 

ss  part 

(i). 

Also,  any  solution  X   must  satisfy  the  relati 

.on  X2E0, 

and  hence,  must  satisfy  either  X^O  or  X  =  0.   It 

follows 

that 

either  L^O  or  Xv=  0,   and,  in  particular, 

* 

X,£0 

or  } 

:^  =  0   must  hold.   If  X*  =  0,   then  AITU*>  0, 

since 

IT  * 
AX1U 

+  X,  >0.   This  establishes  part  (ii). 

Theorem  2.5.   The  dual  systems   V^O,   CTV  ^  0 

and 

-CX  2 

*        * 
0,   X&O  possess  solutions  V   and  X   such 

that 
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*      *  m     *  * 

V  -  CX  >0  and   C*V   +  X  >  0. 

Proof:   Let  I  denote  the  m  x  n  identity  matrix  and  apply 
the  results  of   Theorem  2.1  to  the  dual  systems 

(I,C)TV§0  and   (I,C)(,/,X)T=0,   (W,X)T=  0, 

where  W  is  an  ra  x  1  column  vector.   This  application  implies 
the  existence  of  solutions  X*,.W*,  and  V*  such  that 


m 


Cl,C>AViO,      (I,C)(V/-,X')1  =  0,      (W*,X*)T*0,      and 
(±,C;   V     +   (W   ,X   )x>  0.      These  relations  when  simplified,   yield 
(2.Q)     V*fcO,      cV§o,      _cx*=V/*so,      xVo,      V*-CX*>0, 

and  CTV*  +  X*>  0. 

Corollary  2.4-.   The  dual  system 

V  g  0,   CTV  1  0  and  -CX  £  Q,   X  £  0 
possess  solutions  V   and  X   for  which  the  follov/ing 
alternatives  hold: 


(i)  either  G  V  >0  or  X  >  0, 

(ii)  either  CTV*>  0  or  X*2  0, 

(iii)  either  V  >  0  or  -CX*>  0 

(iv)  either  V  >  0  or  -CX*>  0. 

Proof:  Statements  (i)  through  (iv)  are  immediate  consequences 
of  relations  in  statement  (-2.9).  It  should  be  noted,  however, 
that  from  the  first  and  third  relations  of  (2.9),  it  follows 

t   0*(CTV)TX=VT(CX)£0,   and  hence,   VTCX=0  for  all 
solutions   V   and  X*.   Therefore,  the  alternatives   (i) 
through  (iv)  are  mutually  exclusive. 


Theorer;;  2.  A.   The  general  dual  systems 
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(U  unrestricted) 

V  £0 

m  rn 

.,-;;  +  crvfco 


-AX   -  EY  =  0 
-CX   -  DY^O 

x£o 

(Y  unrestricted) 


possess  solutions  U  .,  V   and  X  ,  Y   such  that 


*        *        *  rn        m  *       * 

V   -  CX  -  DY  >0  and  AXU  +  C*V  +  X  >  0. 


Proof:   Applying  the  results  of  Theorem  2.3  to  the  dual  systems 


n 


*0, 


-A  -B   3 

A   3  -3 

L  C  D-D 


T1  r 


Up 

c   j 


*0  and  - 


A 

-3      3 

X 

X  " 

_._ 

3   -3 

Y-, 

*o, 

V 

2 

C 

J   -3 

Y^ 

.   -J 

, 

JO, 


where  U-,   and  Up  are   m  x  1   column  vectors,  and  where 
Y,   and  Yp  are  n  x  1  column  vectors,  in. plies  that  there 
exist  solutions 


U£<rO,      Up  5  0,      V'fcO     and      X    S.O,      Y,fcO,      Yp5  0, 


uch  that 


rn     *  rn     w  ■"•      * 

-A-U1    +    A^U0   +    (TV    £  0 
-3 1U1   +    3XU2   +    3XV    *  0 

3iu1  -  iru?  -  3xy  50 

V*-   CX*  -    DY*   +    DY*  >0 


AX*    +   .bY*_   -  BYp  5  0 

-AX*    -   BY*    +  3Y*5  0 

-OX*    -   DY*_   +  DY*S0 

m     *  rn     *  .rn     *  * 

-A^U,    +   AiTJp  +   CTV      +   X    >  0. 


Choose      U   =  U]    -   Up     and     Y  *  Y, 


-   i0.      Then, 
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m     * 

A-U 

+  C1V  £  0 

-AX*-  BY*=  0 

B1U 

m   * 

+  zrv  =  o 

*         * 

-CX  -  2Y  g  0 

m   *        * 
r   C   V     +   A   > 

0 

V*  -  CX*-  DY*>  0 

Therefore 

,  the  vectors 

* 

u 

* 
and  Y   are  the  desired  solutions. 

Let 

the  matrix   C 

of 

Theorem  2.3  he  a  skew-symmetric 

trix,  i 

m 

.e.,   C  --C. 

The 

n,  the  dual  systems  of  Theorem  2.3 

become   v 

SO,   CViO  a 

nd 

X^O,   CX  t  0  with  solutions 

* 
V 

* 
and  X 

* 
such  that  V  - 

* 
CX 

*         * 

>  0  and  X  -'  CV  >  0.   Since 

these 

two  dual 

systems  are  th 

e  sa 

me,  the  systems  corresponding 

to  a 

skew-symn. 

etric  matrix  i 

S  CI. 

lied  self-dual. 

Theorem  2.5.   The 

self 

* 
W 

-dual  system  KW£  0,   W£0  w 

*    .  * 
such  that  KW  +  W.  ">0. 

here 

T 

has  a  solution 

Proof:   T 

. e  proof  invol 

ves 

an  application  o£   .the  results 

of 

Theorem  2 

.3  to  the  skew 

-syn 

T 
metric  matrix  C-K  —-K.      Ace 

ording 

to  Theore 

m  2.3,  there  exist 

*        * 
solutions.  V   and  Irf   such 

that 

* 

0,   KV*S  0,   KX** 

0,   X*S  0,   V*  +  KX*>  0,   KV* 

* 
+  X  >  0. 

Hence , 

■ 

K(V* 

+  X*)fcO,   (v 

* 

+ 

X*)£0,  and  K(V*  +  X*)  +  (V* 

+  X*)>0. 

.  If  W*   i 

s  taken  to  be 

* 
V 

* 
+  X  ,   the  Theorem  follows. 

More 

specifically, 

the 

*         * 
two  vectors  W   and  KV/   have  the 

property 

that  for  each 

J  = 

l,...,n,   the  j-th  component 

of   one 

vector  i 

s  positive  if, 

and 

only  if,  the  j-th  component  of  the 

other  is  zero.   Indeed,  Theorem  2.5  shows  their  sum  is  positive, 

but  their  inner  product   (W  )  (KW  )   is  zero,  since  the  transpose 
of  (W  )  KW   is  its  own  negative. 

Definition.  A  slack  inequality  in  a  system  of  inequalities 
is  an  inequality  (fc  0)  which  is  satisfied  as  a  strict  inequal- 
ity C**  0)   by  some  solution  of  the  system. 

Now,  consider  the  system  A  U^O,   b  U~  0  of  Theorem  2.2. 

Let  U   be  a  solution  of  this  system.   It  may  happen  that 

T  T 

A±U>0  for  some   i  =  l,...,m.   That  is,  A£U  fe  0  may  be  a  slack 

T1 

inequality  for  some  row  AT.   If  there  does  exist  a  .solution  U 

such  that  AiU2?0  is  satisfied  as  a  strict  inequality,  let  this 

solution  be  denoted  by  bh,  i.e.,   A?IL>  0.   Now,  let   I   denote 

the  set  of  indices  i  such  that  A?Ui>  0  for  some  solution  U 

of  the  system.   Then  Zluh,  summed  for  all   i   in   I,   is  a 

solution  of  the  system.   Indeed,   AT(  "ZIU.)  =  0  and  BT(  5~  U  )  =  0 

i  —  i 

since  each  ■  bh   for  i   in  I  is  a  solution  of  the  system. 
Furthermore,   A;r{  EU^O  for  all  i  in  I  since  at  least 
one  of  the  summands  is  positive.   This  shows  that  the  slack  in- 
equalities in  a  system  can  be  characterized  collectively  as  the 
maximum  set  of  inequalities  of  .the  system  which  are  satisfied  as 
strick  inequalities  by  some  solution  of  the  system.   Hence,  the 
remaining  inequalities  in  the  system  are  those  which  are  satisfied 
as  equations  by   all  solutions  of  the  system. 
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Theorem  2.6.   In  the  general  dual  systems 

(2.10)   (U  unrestricted)         -AX  -  BY  =  0 
V?C  _CX  -  DY  £  0 

ATu  +  crv  |o  x ^  o 

B  U  +  j*Vsq  (Y  unrestricted), 


of  Theorem  2.4,  each  of  the  m  +  n  pairs  of  corresponding 
inequalities 


b 


^ahjuh  +Scijvi=°  *i^°   (J  =  1 n) 


contains  exactly  one  inequality  that  is  slack  relative  to  the 
system. 

Proof:   Let  U,  V  and  X,  Y  be  any  solution  of  the  given  dual 
system,   Then,  by  multiplying  together  corresponding  terms  in 
bhe  dual  systems  and  summing,  the  relations 

(2.12)  UT(-AX  -  BY).  -UTAX  -  UTBY  =  0, 

(2.1^)  VT(-CX  -  DY)=  -VTCX  -  ^DY*0, 

(2.14)  (ATU  +  CTV)TX  =  UTAX  +  VTCX*0, 

(2.15)  (3TU  +  DTV)TY  =  UT3Y  +  VTDY=0, 

are  obtained.   Adding  (2.12)  to  (2.14)  and  (2.13)  to  (2.15), 

one  obtains  -UTBY  +  VTCX  >  0  and  UT3Y  -  VTCX£o'. 

hence,   UTBY  -  VTCX = 0.   Using  this  result,  equations  (2.12)  and 

(2.15)  yield  the  equations, 

(2. 16)  -UTAX  =  UT3Y  =  VTCX =  -VTDY.  ■ 
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ibstituting    from   (2.16)    into   (2.13)    -'-nd   (2.14),    it   is   evident 


(2.17)         v-(-o:<  -  m.  ?  v.  f  Ci jX .  .  Z  dikyk).0> 

Equation  (2.17)  shows  that  in  each  pair  of  corresponding  in- 

alities  of  (2.11),  at  least  one  sign  of  equality  must  hold 
for  all  solutions  of  the  systems;  otherwise,   V(-CX  -  DY)'^0 
or   (A  +  C  V;  X^O  for  some  solutions.   Therefore,  each 
pair  of  dual  inequalities  in  (2.11)  contains  at  most  one 
inequality  that  is  slack. 

By  Theorem  2.4,  there  exists  solutions  U*,  V*   and   X* 
Y   of  the  dual  systems  such  that 


,m), 


V  e?ci^r?^kyk>+>0       ci.if. 

(¥  a- JUn    f  |Icijvi)    +   xl>°  (j  =  l,...,n). 

Hence,  each  pair  of  corresponding  inequalities  contains  at 
3t  one  inequality  that  is  slack. 

Taken  together,  the  last  two  paragraphs  imply  that  each 
pair  of  corresponding  inequalities  in  bhe  general  dual  systems 
(2.10)  contains  exactly  one  inequality  that  is  slack  relative 
to  the  system. 

The  property  of  the  system  (2.10)  exhibited  in  Theorem 
2.6  can  be  described  collectively  as  complementary  slackness,  i.e., 
the  set  ox  sxacic  inequalities  in  the  one  system  is  exactlv 
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complementary  to  the  set  of  slack  inequalities  in  the  other  system. 
The  property  o.f  complementary  slackness  applies,  of  course,  to 
fc]  e  pairs  of  dual  systems  in  Theorems  2.1,  2.2,  and  2. 3.   Indeed, 
Theorem  2.6  can  be  reduced  to  these  pairs  of  dual  systems  by 
letting  certain  of  the  matrices  A,  3,  C,  and  D  become  vacuous. 
In  particular,  this  property  shows  that  the  alternatives  in  the 
various  parts  of  Corollaries  2.2,  2.j>,    and  2.4  are  mutually 
exclusive. 

Theorem  2.7.-  In  the  self-dual  system  KW£0,   W^O, 

m 

K  a  -K;  each  of  the  n  pairs  of  corresponding  inequalities 

*FkijW0~°  an(i  wi~  °'     ^  =  3->--->n),   contains  exactly 
one  inequality  that  is  slack  relative  to  the  self-dual  system. 

Proof:   For  any  solution  W  of  the  given  system,   WTKW-0, 
because   tf  KV«'s  -W  KW.   Hence,  in  each  of  the  -airs  of  corre- 
sponding inequalities  21k.  .w.^0  and  w.^0,   (i  =  l,...,n), 
at  least  one  sign  of  equality  must  hold  for  all  solutions   ./. 
Therefore,  each  pair  of  cox^responding  inequalities  contains  at 
most  one  inequality  that  is  slack. 

By  Theorem  2.5,  there  exists  a  solution  W*   such  that 
?-".  k..w.  +  w*  >0,   (i  =  l,...,n). 

Hence,  each  pair  of  corresponding  inequalities  of  the  self-dual 
system  contains  exactly  one  inequality  that  is  slack  relative 
to  the  self-dual  system. 

It  should  be  emphasized  that  a  system  of  inequalities, 
AX£B,  in  which  are  included  the  inequalities,  X,  £0,..., 
Xn§  0,  restricting  the  unknown  of  a  solution  to  nonnegative 
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values  is  necessarily  n  :e  no  linear        ce 

of  column:  brix  A  can  occur.   Therefore,  linear  pro- 

i-iiij  C0]        i       'mally  formulated,  are  nonsingular 
systems.   However,  the  solution  of  bhe  basic  system  of  inequali- 
ties may  be  viewed  in  two  ways:   (1)   to  solve  a  system   (A,I)X4B 
of  m  +  n  linear  relations  in  n  unknown,  or  alternately,  (2) 
to  find  nonnegative  solutions  of  a  system,   AX£  B,   X^O,   of 
m  linea  relations  in  n  unknowns.   The  problem  has  been  studies 
in  this  section  from  the  latter  viewpoint. 
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Duality  and  Existence  Theorem 


Two  or  the  fundamental  results  in  linear  programming  theory 
are  the  Duality  Theorem  and  the  Existence  Theorem.   The  essential 
steo  in  the  derivation  of  these  theorems  is  the  use  of  Theroem 
2.5  to  p-.v;e   z:;o   lemmas  that  lead  at  once  to  the  basic  theorems 

ut  the  aual  pair  of  linear  programming  problems  (1.7)  and 
(l.o): 


(3.D- 

(3.2) 


Maximize  CX,  'subject  to  AX<B,   X^O. 
Minimize  UB,   subject  to  UAfcC,   U^O. 


The  first  step  is  to  use  matrices  A,  3,  and  C   to  con- 
struct a  skew-symmetric  matrix  of  order  m  +  n  +  1: 


K- 


0  -A   B  ' 

AT  0  -CT 
-3T  C   0  J 


-KT. 


3y  Theorem  2.5,  there  is  an  (m  +  n  +  1)  x  I  column  vector 
=  (Uo'Xo'VT*°>   where   tft   is 


(3.5) 


.s  some  scalar,  such  that 
'■:  *  0,   KW^O,   and  KW  +  W  >0. 


When  the  expressions  for  K  and  W  are  substituted  in  (3.3), 
the  following  six  inequalities  in  nonnegative  variables  are 

obtained: 
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(3.*) 

(i) 

(ii) 

(iii) 

(iv) 

(v) 
(vi) 

.AX  *  Bt 

o    o 

U  A  5  t  C 
o    o 

CX  ^  BU 
o     0 

kX<  fcB  f  l/ 

0     O       0 

m      rn  rn 

t  c^<  A^-uj;  +  xo 

0        0     o 

m  m 

B^U^<  CXrt  +  t  . 

0      O      0 

Throuf 

;hout  th 

is  section,  the  symbols   v.', 

U  ,  X  ,  t  , 
0'   o'   o' 

and  ine 

qua] 

.iti<  s  ( 

i)  through  (vi)  will  have  t 

he  meanings  just 

desij ;;.:- 

ted. 

.•o  a 

lternatives  occur  in  (3»4): 

either  t  >  0 

0 

or  t„. 

:  0. 

..'he  the 

r  or  not  the  systems  .(3.1) 

and  (3.2)  have 

optj 

sol 

.utions 

depends  on  whether  t  >  0 

0 

or  t  =  0.   The 

0 

effects 

of 

these  t 

wo  possibilities  will  be  investigated  sepa- 

rately 

in  Lemmas  3 

.1  and  3.2. 

Lemma 

?.l.   I 

f   t  >  0,  then  the  dual  pro 

grams  (3.1)  and 

(3.2)  h 

ave 

optimal 

vectors  X   and  U°   such 

that  CX°=U°B, 

(U°)T  t 

3  > 

AX°,   and  U°A  +  (X°)T>C. 

Proof: 

Sir 

:ce   tQ> 

•  0,  the  nonnegative  vector 

.■/  =  (U  ,XT,t  )T 
v  o'  o'  oy 

car.  be 

normalized 

so  that  t  =  1  without  aff 

ecting  the  validity 

of  re la 

tioi 

is  (J.4) 

Then,  (i)  and  (ii)  with 

t„-  1   show  that 

c 

0   0 

arid 

V^o 

are  feasible  vectors  for  t 

heir  respective  pro- 

grams, 

while  (iii) 

and  Lemma  1.1  show  that  C 

<  V*o> '  <  VVB- 

Hence, 

Lemn 

ja  1.2  implies  that  X  /t   and  U 

0   0           0 

/t   are  optimal 

solutions. 

Thus, 

one  can  choose  X  /t   and 

o   o 

UQ/t   to  be  the 

desired 

vec 

;tors  X 

and  U°.   The  strict  ine 

qualities  of  the 

Lemma  follow  respe 

ctively  from  (iv)  and  (v)  w 

ith   t  =  1. 
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Lemma  y.2.      Suppose   t  =0.   Then  at  least  one  of  the  dual 
programs  has  no  feasible  vector.   If  the  maximization  problem 
has  a  feasible  vector,  then  the  set  F  of  all  feasible  vectors 
is  unbounded  and  bhe  function  CX  is  not  bounded  above  on  F. 
Dually,  if  the  minimization  problem  has  a  feasible  vector,-  then 
?  is  unbounded  and  UB  is  not  bounded  below  on  P.'  Neith* 
of  the  dual  problems  has  an  optimal  vector. 

of":   Lssume  that  X  and  U  are  feasible  vectors  for  the 
maximization  and  minimization  problems  respectively.   With 
t  s  0,  inequalities  (i),(ii),  and  (vi)  read  AX  &.0,      0 i  U  A, 

CX  >  U  B".   It  follows  from  (ii),  (vi),  the  fact  that 
UqS  0,  and  the  maximization  problem  constraints  AX^B,  that 

C3.5)      O^U0AXSU0B<CXo. 

Now,  with  the  aid  of  (i),  the  fact  that  XQ^  0,   and  the  minimi- 
zation problem  constraints   UA5C,   it  follows  that 
0lUAXo5CXo.   This  clearly  contradicts  (2.5).   So,   t  -   <D 
precludes  the  possibility  that  both  of  the  dual  programs  are 
feasible.   Sometimes,  neither  program  is  feasible. 

Now,  the  situation  where  one,  and  hence,  only  one,  of  the 
dual  programs  is  feasible  must  be  studied.   Suppose,  for  example, 
that  X   is  a  feasible  vector  for  the  maximization  problem.   To 
prove  the  second  statement  of   the  Lemma,  one  must  examine  the 
infinite  ray  consisting  of  the  vectors  X  +  rX  ,   where  r  is 
a  positive  scalar.   It  is  a  straightforward  calculation  to  show 


that      X  +  rXQ   are  feasible.   Indeed,   X  +  rXQi=  0,  and  using  (i) 
with  t0  =  °'   il  is  evident  that  A(X  +  rX  )  §  AX  ^  B.   Thus, 
":-°  entir     'inite  ray  consists  of  feasible  vectors.   Further- 
more, the  fact  that   CXQ>  0,   as  was  s'  own  in  (3-5),  makes  it 
clear  that  C(X  +  rXQ)=  CX  +  rCXQ  can  be  made  arbitrarily  larg 
by  choosing     r  large  enough.   Thus,  the  maximization  problerr 
wit}:  a  feasible  solution  has  an  infinite  number  of  feasible 
solutions,  but  has  no  maximum  when  tQ»  0.   An  analogous  discus- 
sion shows  that  if  the  minimization  problem  is  the  only  feasible 
one,  then  the  desired  minimum  is  not  attainable. 

In  summary,  if   tQ  =  0,   then  at  least  one  of  the  dual 
programs  is  not  feasible  and  neither  has  an  optimal  solution. 

£££°ll§£XJLJki  Either  both  the  maximization  and  minimi- 
zation problems  have  optimal  vectors  or  neither  does.  In  the 
first  case,  the  attained  maximum  and  minimum  are  equal;  their 
common  magnitude  is  called  the  optimal  value  of  the  dual  pro- 
grams. 

Proof:   If  one   of  the  dual  programs  has  an  optimal  vector,  then 
the  last  statement  of  Lemma  3.2  implies  that   t  >  0,   and  then 
Lemma  3.1  implies  that  both  programs  have  optimal  vectors  X° 


.0 


and   U    such  that  the  maximum   GX°   is  equal  to  the  minimun 


U°B. 


9^olj-ai-y_2^2_L  The  dual  programs  have  an  optimal  value  if, 
and  only  if,  either  CX  or  UB  is  bounded  on  the  corresponding 
nonvacuous  set  of  feasible  vectors. 
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Proof:   the  necessity  of  the  condition  is  clear.   To  prove  the 
sufficiency,  suppose  that  the  maximization  problem  has  a  non- 
vacuous  set  ?  of  feasible  vectors  and  that  CX  is  bounded 
on     ?. 

The  second  statement  of  Lemma  3.2  implies  that  t  >  0.  Thus, 
by  Lemma  3.1,  both  of  the  dual  programs  have  optimal  vectors, 
and  according  to  Corollary  3.1,  they  have  an  optimal  value. 

Theorem  ;>.!  (Duality  Theorem).   A  feasible  vector  X° 
for  the  maximization  problem  is  optimal  if,  and  only  if,  there 
is  a  feasible  vector  U   for  the  minimization  problem  with 
U  3= OX  .   Alternately,  a  feasible  vector  U°   for  the  minimiza- 
tion problem  is  optimal  if,  and  only  if,  there  is  a  feasible  vector 
X   for  the  minimization  problem  with  CX°=U°B.- 


Proof:   To  prove  the  necessity  of  .the  first  stated  condition, 
suppose  that  X   is  an  optimal  vector  for  the  maximization  prob- 
lem.  Then,  the  last  statement  of  Lemma' 3.2  implies  that   t  >  0. 
Thus,  by  Lemma  3.1,  the  minimization  problem  also  has  an  optimal 
vector.   According  to  Corollary  3.1,  the' attained  maximum,   CX° 
and  the  minimum  U°B  are  equal.   That  the  first  stated  condi- 
tion is  also  sufficient  follows  immediately  from  Lemma  3.2. 
Proof  of  the  second  statement  of  the  Theorem  is  similar. 

Theorem  3.2  (Existence  Theorem).   A  necessary  and  suffi- 
cient condition  that  one,  and  hence  both,  of  the  dual  problems 
have  optimal  vectors  is  that  each  px^oblem  has  a  feasible  vector. 
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Proof:   The  necessity  of  the  stated  condition  is  evident  from 

the  Duality  Theorem. 

If  each  of  the  dual  problems  has  a  feasible  vector,  it  follows 

from  the  fir.ru  statement  of  Lemma  3.2  that   t  >  0.   If  t  >  0. 

o  o    ' 

Lemma  5.1  implies  the  existence  of  optimal  vectors  X°  and  U°. 
This  proves  the  sufficiency  of  the  condition. 


The  following  corollary  deals  with  the  way  in  which  a  pair, 
X  and  U  ,  of  optimal  vectors  satisfy  the  2m  +  2n  feasibil- 
i  ty  c  ons braints 

AX  £B,   X  * 0  and  UA  fc  6,   U  ^  0. 

There  is  a  convenient  way  to  pair  off.  the  constraints 
associated  with  the  two  dual  problems.   Let  the  i-th  relations 
in  the  system  AX  SB,   U|0  be  called  a  pair  of  dual  constraints, 
as  well  as  the  j-th  relations  in  the  system  UA^C,   X2  0. 

Corollary  ^..ft.   If  both  the  minimization  and  maximization 
problems  have  feasible  vectors,  then  they  have  optimal  vectors 
such  that  the  dual  constraints  are  complementary  slack  inequali- 
ties.  That  is,   m  +  n  on  the  constraints  are  satisfied  as 
equations  and  the  remaining  m  +  n  constraints  are  satisfied 
a.i  strict  inequalities. 

Proof:   Theorem  3. 2  implies  that  both  problems  have  optimal 
vectors.   The  last  statement  can  be  seen  by  substituting  appro- 
priately into  the  general  dual  system  of  Theorem  2.6.   To  relieve 
the  problem  of  nomenclature,  let  the  two  systems  of  the  Corolla ■■ 
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(3.6)      A'XJB',   U'^0   and-  XfeO,   U'A'^C. 

Then  in  the  general  dual  system  of  Theorem  2.6,  since  vectors 
U  and  Y  are  unrestricted,  choose   U,  Y,  A,  and  U   such  that 
-UTA=C   and  -JY=B'.   Let   v=(U')T  and  C=Af .   After  making 
br.ese  substitutions,  it  is  readily  seen  that  the  m  +  n  pairs 
of  inequalities  referred  to  in  Theorem  2.6  are  those  in  (3.6). 
Hence,  each  of  the   m  +  n  pairs  of  dual  constraints  contains 
exactly  one  inequality  that  is  slack. 

Corollary  3«3  may  be  stated  more  explicitly  so  that  its 
implications  will  be  readily  available  for  use  in  later  proofs. 
The  restatement  will  require  the  following  two  corollaries. 

Corollary  3.4.   If  both  of  the  dual  problems  have  feasible 
vectors,  then  they  have  optimal  vectors  X°  and  U°  such  that 
if  X   satisfies  a  row  constraint  as- an  equation,  then  U° 
satisfies  the  dual  constraint  as  a  strict  inequality;  and  if  U° 
satisfies  a  column  constraint  as  an  equation,  then  X°   satisfies 
the  dual  constraint  as  a  strict  inequality. 

Proof:   By  Theorem  3.2,  both  problems  have  optimal  vectors  so 
that  Lemma  3.2  implies  that   t  >  0.   Therefore,  Lemma  3.1  implies 
that  there  exist  optimal  vectors   X°   and  U°   such  that 
(U0)1  +  3>AX°  and  U°A  +  (X°)-L>C.   The  vectors  thus  exhibited 
have  the  required  properties. 

Corollary  3.3.   If  both  of  the  dual  problems  have  feasible 
vectors,  then  for  any  row  index  i,  either   (AX°).<b.   for  some 
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optimal  vector  X   and  u.  =0  for  every   optimal  vector  U,   or 

(AX).  =  b.   for  every  optimal' vector   X   and   u.  >0   for  some 

'i  1  J      1  i 

imal  vector  U  .   The- dual  statement  is  also  true. 


i'rcof:.  By  Theorem  5.2,  both  of  the  dual  problems  have  optimal 
vectors.   Corollary  5.4  implies  that  if  (AX).  =  b.   for  every 
optimal  vector  X,   then  u? >  0   for  some  optimal  vector  U°. 
However,  if   (AX)^  bi,   then   (AX°) . <  b.   for  some  optimal 

vector  X  .   To  complete  the  proof,  it  must  be  shown  that 
U.  =  0  for  every  optimal  vector  U.   Assume  this  not  ture, 
i.e.,   u.  >  0  when   (AX  )i<-'^i   for  some  optimal  vector  X  . 
This  implies  that  u.(AX). <  u.b.   for  some  index  i.   Since   X 
is  feasible,   CX°£  UAX°,   or  in  particular,   (CX°) . 5  u.(AX°) . . 
:;ence,   (CX  )i<u.b..   But,  Corollary  3.1  implies  that 
(CX  ).  »  uibi   since   X°   and  U  are  optimal  vectors.   Thus, 
the  contradiction  implies  that   u.  =  0   for  every  optimal  vector 
U. 
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Systems  With  Mixed  Constraints 

Linear  progra mining  problems  involving  mixed  constraints 
(both  equalities  and  inequalities)  frequently  arise  in  practice. 
A  method  due  to  A.  v.'.  Tucker  (8),  for  reducing  such  a  problem  to 
canonical  form  is  presented  in  this  section. 

Let  N  be  the  set  of  indices   (1,2,..., n),   and  let  M 
be  the  set  of  indices   (l,2,...,m).   Suppose  that  N-j   and   IT? 
are  complementary  subsets  of  N  with  n,   and  n2  elements 
respectively,  and  that   M^  and  M~  are  complementary  subsets 
of  M  with  m1  and"  m0  elements  respectively.   The  dual  linear 
programming  problems  with  mixed  constraints  that  will  be  investi- 
gated in  this  section  may  now  be  stated: 

(4.1)  Maximize   0^+...+  cnxn  subject  to  the  constraints 

ailxl+'"+  ainxn"bi   for  each  i   in  M., 

ailxl+ ' * '  +  ainxn  =  bi  for  each  i     in  •  M2» 
x.SO   for  each  j   in  W,, 

x.   unrestricted  for  each  j   in  N0. 

J  c 

(4.2)  Minimize  u^+.-.h-  unbn  subject  to  the  constraints 

ulalj  +'-'+  umamj«cj   for  each  J   in  Ni» 

.  Ulalj  +   +  umamj = cj   for  each   J   in  N2» 
ui^0   for  each  i   in  M,, 

ui   unrestricted  for  each  i   in  T-i0l 

Note  that  the  constraints  which  are  equations  correspond 
to  variables  -/hich  are  unrestricted. 
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If  the  matrix  A  and  the  vectors  B,  C,  X,  and  U  are 

par 

titioned  into  clocks  corresponding  to  the  decomposition  of 

.'. 

and  N  into  M,  +  KP  and  N,   +  Mp  respectively,  then, 

the 

problems  (4.1)  and  (4.2)  may  be  written  in  matrix  form  as 

To  How  3: 

Maximize  C,X,  +  CUX0  subject  to  the  constraints 

A11X1  +  A12XP  =B1 

A21X1  +  A?2X?  =  32 

X,£0,  X.,  unrestricted. 

Minimise   U-.B..  +  tUB^   subject  to  the  constraints 

U1A11  +  U2A21^C1 

'U']_A]_2  +  '-p-^p-^2 

U,  5  0,   Up  unrestricted. 

The  definition  of  feasible  and  optimal  vectors,  as  pre- 

vio 

usly  riven,  ap  Ty  -.3   wel]  to  the  more,  general  problem  in  this 

sec 

tion.   That  is,   X  or  U  is  feasible  if  it  satisfies  the 

con 

straints  and  is  optimal  if  it  is  feasible  and  achieves  the 

d^s 

ired  maximum  or  minimum. 

It  should  be  emphasized  that  the  problems  (4.1)  and  (4.2) 

are 

essentially  no  more  general  than  the  problems  (1.7)  and 

(l.J 

3) »  which  are  in  canonical  form.   Indeed,  (4.1)  and  (4.2) 

reduce  to  (1.7)  and  (1.8)  respectively,  if  the  sets  M.D   and 

are  vacuous.   Furthermore,  the  metiu>u  of  replacing  each 

equation  of  a  system  by  two  inequalities  and  each  unrestricted 

variable  of  the  system  by  the  difference  of  two  nonnegative 
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variable.',  results  Ln  prol  Lems  in  canonical  form.   However,  the 
new  maximization  problem,  Tor  example,  will  have  m'=2m  -  ~ 
i:: equalities  and  n'=2n  -  f^  nonnegative  variables,  where 
lo   denotes  the  number  of  elements  in  Mx  and  FT.   denotes 
the  number  of  elements  in  N, . 

The  method  of  the  preceding  paragraph  can  be  implemented 
formally  by  first  setting. 

(4.3)      Xi=max(0,X2)       U2=max(0,U2) 
X^amax(0,-X2)      Ug«max(0,-U2) 

where  maxCY1  ,  Y2)   is  the  vector  whose  components  are 
•i-naxCyJ.yf).  Prom  (4. J),  it  follows  that  XP»xl  -  x"  and 

U?=U2  -  U2.   Secondly,  each  equation  E=G  is  replaced  by 
the  equivalent  pair-  of  inequalities,   3^0  and  -E§0. 
After- performing  these  two  steps,  the  problems  (4.1)  and  (4.2) 
will  be  reduced  to  canonical  form  and  may  be  stated  as  follows: 

0.4)      Maximize   C^  +  C^  +  C-C2)j£   subject  to  the 
constraints 


A11X1  +     A12X2  +  ("Ai2)X2  =B1 

A21X   +    h?4   +  C-A22)X2?  t  B2 

(-A21)X1  +  (-A22>X2  +    A22X2  i(-B2) 

x1^o,  x2^o,  x2so. 
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(4.5)     Minimize  U1B1  +  u2B2  +  U2(-B2)   subject  to  the 
constraints 

U1A11      +  U2A21  +  V"A2l)  Cl 

U1A12       +  U2A22  +  U2(-A22)  C2 

U1(-A12)    +  U^C-Agg)  +  U2A22  ^~C2^ 
ux  0,  u2  0,  u2  0. 

As  previously  noted,  the  dual  problems  (4.4)  and  (4. 5) 

contain  :,ore  constraints  than  the  dual  pair  (4.1)  and  (4.2). 

however,  the  two "problem  pairs  are  equivalent  in  the  sense  that 

•   it 
(U1,U2,U2)   is  feasible. (optimal)  if,  and 'only  if,   (l^Up)  is 

feasible  (optimal),  and  similarly  for  the  dual  problem.   There- 
fore, the  Duality  and  Existence  Theorems  of  the  last  section  are 
valid  for  the  systems  (4.1)  and  (4.2). 

An  al  orithm  for  reducing  a  pair  of  linear  programming 
problems  with  mixed  constraints  to  canonical  form  will  now  be 
given.   This  method,  mentioned  in  the  opening  paragraph  of  this 
section,  does  not  have  the  disadvantage  of  the  reduced  canonical 
form  containing  more  inequalities  than  the  original  problem. 
The  method  eliminates  the  equalities  and  unrestricted  variables 
while  the  sum  of  che  number  of  inequalities  and  the  number  of 
nonnegative  variables  remains  constant  under  the  reduction.   The 
method  utilizes  the  fact  that  any  equation  of  the  system  con- 
taining a  variable  with  non-zero  coefficient  can  be  used  as  a 
defining  relation  to  eliminate  that  variable  from  the  system. 

Suppose,  for  example,  that  the  m-th  constraint  is  the 
equation  a^  +   +  a^-b^  with  amn  0.   That  is  m  is  in 
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M*  ana  the  variable  U   is  unrestricted.   ?Tow,  two  situations 
c  m  occur;  either  n   is  in   N»   or  in  Np. 

If  n  is  in  Np,   the  n-th  column  constraint  is  an  equa- 
tion and  the  variable  xn  is  unrestricted.   Then,  the  variables 
xn  and  um  can  be  eliminated  from  the  systems  by  the  relations 


mn       ml  1      m,n-l  n-ly' 


"m=  ^(CJ  "  Ul&ln Um-lam-l,r> 

Mheii   these  variables  are  eliminated,  the  elements  a..,   b. 
and  c.   will  be  replaced  by 

(4.?)       a.  .=a.  .  -  \fi* 
ij •  ij     a     ' 
u    °  mn 

b  a. 

b>b.  -   n'  in, 
11    a    ' 
mn 

V°j  "  -f1^'   for.  i=l,-v.,m-l;  j=l,...,n-l.   ■ 

u   u     mn 

Npte  that  the  m-th  row  constraint  and  the  n-th  column  constraint 
will  also  be  eliminated. 

The  new  dual  problems,  given  by  the  matrics  of  lower  decree 
which  contain  the  elements  ('(-.7),  are  equivalent  to  the  old 
problems,  (4.1)  and  (4.2;,  in  the  sense  that   (x,,...,x)'  or 
Culf...,um)   is  feasible  (optimal)  for  the  old  problem  if,  and 
only  if,   (xx,..  .,xn_1)  •  or  (.U;L, ... .  ,ud-1)  is  feasible  (optimal) 
for  the  new  problem. 

If,  however,  n  is  in  r^,   the  n-th  column  constraint  is 
an  inequality  and  the  variable  xn  is  nonnegative.   Then,  by 
cing  the  same  substitution  for  x   as  given  by  (4.6) 
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replacing  the  m-th  equation  by 

_ml  x,  +  ...+  am,n-l  xn  ,  K    °m  , 

amn •        *  amn 

and  the  unrestricted  variable  u   by  the  nonnegative  variable 

u  =u-,a.,   +...+  u  a   -  e  , 
m  1  In        m  mn    n' 

a:,  equivalent  problem  is  obtained.   The  maximization  problem 
has  one  ;;iore  inequality  and  one  less  nonnegative  variable  while 
in  the  minimization  problem,  an  inequality  has  been  replaced 
by  demanding  that  another  variable  be'  nonnegative. 

Again,  the  new  dual  problems  obtained  have  feasible  (optimal) 
vectors   (x1,  .  .  .  ,xn-1)   and   (i^  ,  .  .  .  ,um_1  ,u"Tfl)   if,  and  only  if, 
(x1,...,xn)   and   (u],...,un)   are  feasible  (optimal)  for  the 
old  problems.   In  either  case  (n  in  PL   or  in  Np),  the  func- 
tions CX  and  U3  are  changed  by  the  same  constant  b  c  /a 

.  _  m  n   mn 

Evidently,  a  similar  elimination  reduces  the  number  of 
equalities  and  unrestricted  variables  when  applied  to  the  column 
constraints  that  are  equations.   After  a  finite  sequence  of  such 
operations,  applied  to  both  rows  and  column,  one  reaches  a  situation 
in  which  all  constraint  equations  (if  any  exist)  have  only  zero 
coefficients.   If  all  of  the  constant  terms  (the   b.'s   and 
cj's)   i:i  these  equations  are  zero,  then  the  associated  rows 
and  columns  of  zeros  may  be  deleted.   This  yields  a  problem- 
pair  in  canonical  form,  i.e.,   m?«np'«0. 

If  any  of  the  constant  terms  are,  nonzero,  then  the 
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corresponding  member  of  the  original  problem-pair,  is  exhibited 
as  unfeasible. 
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Systems  of  Equated  Constraints 

In  the  theory  of  linear  programming,  many  significant  re- 

sults pertain  to  associated  systems  of  the  canonical  form  sys- 

tems (1.7). and  (1.8).   The  associated  systems  that  will  be  dis- 

cussed in  this  section  contain  only  equations,  namely,  aystems 

of  equated  constraints. 

Let  the  systems  of  inequalities  under  consideration  be 

the  systems  (1.7)  and  (1.8).   Consider  the  various  subsets  of 

the   m  +  n  inequalities  in  each  system  that  ax*e  satisfied  as 

equations.   It  will  be  convenient  to  describe  these  systems  in- 

bhe  following  manner.   Let  M,   and  M2  be  arbitrary  subsets 

of  the  set  M  of  indices  (l,...,m),   and  let  N,   and  N2 

be  arbitrax-y  subsets  of  the  set   h   of  indices   (l,...,n). 

Then,  each  of  the  systems 

(6.1)      (AX)i=bi    for  all   i   in  M^ 

x.=0     for  all   j   in  Np, 

and 

(6.2)      (UA).  =  c.    for  all   j   in  N,, 
J   J 

u.=0     for  all   i   in  M~, 

in   called  a  system  of  equated  constraints. 

With  the  maximization  problem  in  mind,  consider  the  set  5 

of   its  feasible  vectox-s  X.   The  set  S  '  i.-;  defined  by  the  finite 

system  (1.7).   It  is  the  simple  fact  that  an  individual  feasible 

vector  *'.      of  S  must  satisfy  an  individual   inequality   (AX)  .  ^ 

b. 

l 
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of  (1.7)  either  as  an  equation   (AX).=b.,   or  as  a  strict  in- 
equality  (AX)i<bi,   that,  Rives  structure  to   8.   That  is,  each 
feasible  vector  In  the  set  8  may  be  classified  according  to 
which  inequalities  it  satisfies  strictly  and  which  it  satisfies 
as  equations.   This  leads  to  a  partitioning  of  the  set  S   into 
d        uj-sjomu  subsets  S.A   The  subsets  3r./  are  indexed  by 
means  of  the  set   M  of  all  indices   i   that  specify  the. inequal- 
ities that,  are  satisfied  as  equations.   That  is,  the  set  8..' 
consists  of  all  feasible  vectors  X   that  satisfy 


(AX)i«bi    for  each   i   in   w',   and 
(AX)i<bi    for  each   i   not  in  M.  '■. 

It  should  be  noted  that  some,  or  even  all  of  the  sets   SM' 
may  be  vacuous.   Cf  course,  similar  statements  could  be  made 
about  the  dual  problem. 

The  systems  (6.1)  and  (6.2)  are  said  to  dual  systems  of 
equated  constraints  if  Vi±      and  \\?     are  complementary  subsets 
of  M  and  if  N^   and  N2  are  complementary  subsets  of  N. 
If  the  systems  (6.1)  and  (6.2)  are  dual,  the  equations  may  be 
renumbered,  if  necessary,  so  that   M1=(l,...,p)   and 
K1=(l,...,q)   and  the  systems  written  as 


(6.?)      anx1+...+  alqxq  =  b1 


ulall  + 


+  Vprci 


and 


aplxl+ 


+  a^x  =b 

pq  q  p 


u-,a-,   +-  ..  +  u  a   =c 
1  lq        q  oq  q 


Vi'° V0' 


Vl  =  °'-'"Um=0 
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If  a  system  of  equated  constraints,  after,  being  written  in  the 
form  (6.3)  is  such  that  the  coefficient  matrix 


nr 


.a. 


"11  clif-"iq 


>1  *22' 


-•q 


apl  ap2"-apq 


is  square  and  nonsin,  -ular ,  then  the  system  is  said  to  be  a 
nonsinrular  square  system  of  equated  constraints. 

Theorem  5.1.   Feasible  vectors  are  optimal  if,  and  only  if, 
they  satisfy  dual  systems  of  equated  constraints. 

Proof:   To  prove  the  necessity  of  the  condition,  suppose  that  the 
vectors   X  and  U  are  optimal.   Define  M,   to  be  the  subset 
of  indices  contained  in  M  such  that   (AX).=b.   and  define   M 

-L    A.  £ 

to  be  the  complementary  subset  of  M,  i.e.,   MD*M  -  M  .   Then 
by  Corollary  3.5,  the   u^O  for  all   i   in  Mg.   If  .  N,   and 
N2  are  defined  analogously,  then  it  follows  from  the  same 
corollary  that   X..-0  for  all   j   in  Ng.   Therefore,  the  vectors 
X  and  U  satisfy  the  dual  systems  of  equated  constraints  indexed 
by  Mlf  M2,  N-p   and   N2. 

To  prove  the  sufficiency  of  the  condition,  suppose  that 
feasible  vectors  X  and  U  satisfy  the  dual  systems  (6.3).   Then 


UB= 


-bi=£ui(Z:ai.x.)=^(£uiai.)x.=  rfc.x.=   CX. 


Thus,  by  Lemma  1.2,   X  and  U  are  optimal  vectors. 


^3 


Theorem  5.2.   There  exists  a  unique  pair  of  dual  systems  of 
equated  constraints  such  that  each  is  the  maximal  system  of  equa- 
ted constraints  satisfied  b.y  all  optimal  vectors. 

Proof:   Let  M^   denote  the  maximal  set  of  row  indices  such  that 

(AX)i»bi  for  every  optimal  vector  X.   Similarly.,  let  N,   denote 

the  maximal  set  of  column  indices  such  thai,   (UA)  .=c  .   for  all 

J   J 

optimal  vectors  U.   By  Corollary  3.5,  the  row  and  column  con- 
straints indexed  by  elements,  of.  •  M^  and  N,   respectively, 
determine  dual  system's  of  equated  constraints  which  have  the 
desired  maximal  property.   It  is  also  evident  from  Corollary  3.5 
tnat  the  dual  systems  so  determined  are  unique. 

The  concluding  theorems  deal  with  extreme  solution  vectors. 
An  extreme  feasible  vector 'is  a  feasible  vector  which  is  not 
the  mean,   #(X  +  X2)   or  ^(U1  +  U2),   of  the  two  other  feasible 
vectors.   An  extreme  optimal  vector  is  an  optimal  vector  which 
is  not  the  mean,   ^(X1  +  X2)   or  ^(U1  +  U2) ,   of  the  two  other 
optimal  vectox's. 

Theorem  5.3.   A  feasible  vector  X  (or  U) .  is  an  extreme 
feasible  vector-  if,  and  only  if,  it  satisfies  a  nonsingular 
square  system  o£   equated  constraints. 

Proof:   To  prove  the  sufficiency  of  the  condition,  suppose  that 
a  feasible-  vector  X   satisfies  the  nonsingular  square  system 
with  the  associated  matrix  A,.   Assume  that   X   is  not  an 
extreme  feasible  vector,  i.e.,   {^(X1  +  X2),  where  X1   and 
X   are  feasible.   Since,   x.=0,   x*£0,   and  x2  ^  0  for 
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j>p,   x.^Kx1  +  x?)«0.   hence,   x^=x?=0  for  j  >  p.   now,  if 

Y=(y1 , . . . ,y  , . . . ,y  )   is  any  n-dimensional  vector,  define 

]       2 
7=(yi»«»»»y  )•   Then,  since  X   and  X   are  feasible, 

.,,7:1^B1,   A1X2^B]_,   and  ^(X1  +  X2)=B1,   where  B^lbp...  ,bp)r 

Therefore,  A,X  «A,X  »B,.  Since  A^  is  nonsingular,  %  =X  . 

furthermore,   X%=X  ,  since  the  last  n-p  components  of  both 

X   and  X   are  zero.   It  then  follows  that  X=#(X  +  X  )=X  »X 

is  not  the  mean  of  other  feasible  vectors.   Hence,   X   is  an  ' 
extreme  feasible  vector  of  the  given  system. 

To  prove  the  necessity  of  the  condition,  a  nonsingular 
square  system  of  equated  constraints  satisfied  by  a  given  extreme 
feasible  vector  must  be  exhibited.   Suppose  that  X  is  a  nonzero 
extreme  feasible  vector  of  the  system  (1.7).   Let   M'   denote  the 
set  of  row  indices  for  which   (AX)i=bi»   The  set   M'   is  not 
vacuous,  for  the  assumption  that   M'   is  vacuous  leads  to  a 

contradiction.   The  nonzero  vector  X  has  at  least  one  component 

1        2 
x .  >  0  and  the  vectors  X   and  X   obtained  from  X  by  replac- 

ing  x.   with  x.  +  €  and  x.  -  €   respectively,  have   X  as 
J         J  J 

their  mean.   If  M'  were  vacuous,  i.e.,  if   (AX).<  b.   for  every 

1        2 
i,  then  X   and  X   would  be  feasible  for  sufficiently  small 

6>0.   This  contradicts  the  fact  that   X  is  an  extreme  .feasible 

vector.   Hence,   !v: '  is  not  vacuous. 

Now,  let  l\'p  be  the  set  of  column  indices  such  that  when 

j   is  in  Np,   x.=0,   and  let  iM-,   be  the  complementary  subset  of 

N,   the  set  of  all  column  indices  in  the  system  (1.7) •   Note 

that  N,   is  not  vacuous  because   X^O.   Let   A"  be  she  submatrix 
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of  the  matrix  A  obtained  by  deleting  from  A  the  rows  whose 
indices  are  not  in  M'   and  the  columns  whose  indices  are  in  NP. 
Let  X  be  the  vector  X  with  those  zero  components  deleted 
whose  indices  are  in   N2  arid  let  E  be  the  vector  b  with 
those  components  deleted  whose  indices  are  not  in  M' .   Then 
cle-.rl.y,   JM. 

The  linear  independence  of  columns  of  I  can  now  be  easily 
established.   If  this  were  not  true,  there  would  exist  a  vector 
X  r-0  such  that  AT  =0.-  Now,  consider  the  vectors  a\  =7  +£"* 
and  X2=X  -«*._,   which  have  dimension  equal  to  n  minus  the 
number  of  indices  in  N^  '  If  zero  components  are  adjoined  to 
a1  and  X2,   to  obtain  n-dimensional  vectors  X,   and  Xp ,  the 
mean  of  X]_  and  X2   is  X,   while  for'  sufficiently  small  €>  0, 
Xx  and  X2  are  feasible.   This  contradicts  the  fact  that  X 
is  an  extreme  feasible  vector.   Hence,  the  columns  of  A  are 
linearly  independent. 

The  matrix  A  which  has  linearly  independent  columns  is  not 
necessarily  square.   However,  the  number  of  rows  of  A   must  be 
equal  to  or  greater  than  the  number  of  columns  of  A.    This 
can  be  seen  by   noting  that  a  set  of  ^-dimensional  vectors  is 
linear  ,•  dependent  "if  the  set  contains  more  than  .  w ±     vectoi-s. 
Thus,  by  deleting  a  suitable  number  of  rows  of  the  matrix  a, 
a  square  nonsinCular  submatrix  A±      is  obtained.   Since,   SxV5, 
clearly,  the  extreme  feasible  vector  X   satisfies  the  system 
of  equated  constraints  whose  associated  matrix  is  A 

In  the  above  proof  of  the  necessity  of  the  condition,  the 
vector  was  assumed  to  be  nonzero.   If,  however,   X=0,   it  is 
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cleaa  that   X  is  an  extreme  feasible  vector,  if  it  is  feasible. 
The  parenthetical  assertion  of  the  Theorem  can  be  proved  in  a 
similar  manner. 

Theox-em  5.4.   Feasible  nonzero  vectors  X  and  U  are 
extreme  optimal  vectors  if,  and  only  if,  they  satisfy  dual 
nonsingular  square  systems  of  equated  constraints. 

Proof:   The  sufficiency  of  the  condition  follows  immediately 
from  Theorems  5.1  and  5.3.   To  prove  the  necessity  of  the  condi- 
tion, suppose  that  X  and  U  are  extreme  optimal  vectors.   Let 
the  sets  of  indices  M^,'  M2,  N,,  and  N~  be  renumbered,  if 
necessary,  so  that  (AX)i=bi  for  i   in  M1=(l , . . . ,p) , 
(AX)i<bi   for  i   in  M2»(p  +  1, . . . ,m),   (UA).=c.   for  j   in 
a]_=(1,.  .  .  ,q),   and   (UA).>c,   for  j   in  N0=(q+1, .  .  .  ,n) .   That 
the  sets  M^  and  N,   are  not  vacuous  follows  from  the  same 
argument  given  in  the  proof  of  Theorem  5.3.   By  Corollary  3.5, 
x.=0  for  j  >  q  and  u^O  for  i>p.   However,   some  of  the 
x.   with   j  <  q  or  some  of  the  u^   with  i$p   may  also  equal 
zero;  in  this  case,  let  the  sets  of  indices  be  renumbered  again 
so  that   x.>  0   for   lijtqfq,   x.=0   for   j  >  q ,  '  u .  >  0   fc 
l€i€?^p,   and  u^O  for  i  >  p.   Since  X  *  0  and  U^O, 
p>0   and   q>0,  and  hence,   p>0   and   q>0. 

Let   A   denote  the  submitrix  of  the  matrix   A   in  (1.7) 
which  contains  as  elements  the  intersection  of  the  first   p 
rows  and  the  first   q   columns  of  A. 

The  linear  independence  of  the  first   q   columns  of  A" 


.or 
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can  be  established  using  an  argument  similar' to  the  one  used  in 

the  proof  of  Theorem  5«3«   If  the  first  q   columns  of  A  are 

_  + 

linearly  dependent,  there  must  exist  a   q-dimensional  vector   X  ^0 

such  that   hX  =0.   Consider  the  vectors   JL-X  +&*    and  X2=X  -  el* 
where  X  is  the  vector  with  the  last  n  -  q  zero  components  de- 
leted.  If  zero  components  are  adjoined  to   X,   and  1?   to  obtain 
n-dimensional  vectors  X,   and  X~,   the  mean  of  X,   and  X?   is 
X  while  for  sufficiently  small€>0,   X,   and  X~  are  feasible. 
The  fact  that  the  vectors  X-L   and  X^  are  also  optimal  follows 
readily  from  the  relations   CX,  fr  UB,   CXp =  UB,   and 
GX=;^C(X1  +  X2)=UB,   which  are  valid  because   X,   and  Xp  are 
feasible  and   X  is  optimal.   Indeed,   CX   +   UBSCX,   +  CX0=  2UB 
or  CX-jg  UB.   Hence,   CX1=UB,   and  similarly,   CXp=U3.   By  Lemma 
1.2,  the  vectors  X,   and  X~  are  optimal.   Thus,  the  fact  that 
the  vectors  X.^  and   Xp  are  optimal  with  mean  X   contradicts 
the  fact  that  X   is  an  extreme  optimal  vector  and  leads  one  to 
the  conclusion  that  the  first   q   columns  of  A*  are  linearly 
independent.   Similarly,  the  first   p  rows  of  A*  are  linearly 
independent. 

Now,  consider  the  maximal  set  of  rows  and  the  maximal  set 
of  columns  of  A.   Let  the  sets  of  indices  indicating  these  rows 
and  columns  be  renumbered  so  thab  the  first   p'   rows  of 
£  (p's  p)   are  linearly  independent  and  the  first  q'   columns 
of.  ,\  (q1   q)   are  linearly  independent.   Denote  the  submatrix 
which  contains  as  elements,  the  intersection  of  the  first   p' 
rows  and  the  first  o'   columns  of  I     by  A,.   The  last   q  -  q' 
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columns  of  A  are  linearly  dependent  on  the  first  q'   columns 
and,  therefore,  any  linear  combination  of  the  p'   rows  of  A, 
can  be  extended  to  a  linear  combination  of  the   p'  rows  of  A" 
by  virtue  of  this  linear  dependence..  Now,  the   p'   rows  of  A" 
are  linearly  independent.   Therefore,  the  rows  of  A,,  which 
contain  only  parts  of  the  components  of  linearly  independent 
vectors,  must  be  linearly  independent.   Similarly,  the  columns 
of  A^  are  linearly  independent.   Hence,   A,   is  square  and 
nonsingular.   Clearly,  the  vectors  X  and  U  satisfy  the  dual 
nonsingular  square. system  of  equated  constraints  with  associated 
matrix  A, . 

Corollary   5.1.   The  sets  of  feasible  and  optimal  vectors 
of  dual  linear  programs  have  a  finite  number  of  extreme  vectors. 

Proof:   This  statement  follows  immediately  from  Theorem  5.5  and 
5.4. 

It  is  easily  seen  that  Theorem  5.4  provides  a  method  for 
finding  ■  extreme  optimal  vectors.   However,  to  use  this  method, 
one  must  determine  all  square  systems  of  equated  constraints 
with  nonsingular  associated  matrices  and  find  the  feasible 
solutions  of  these  systems.   This  procedure  would,  in  general, 
require  a  tremendous  amount  of  work. 


49 


Optimal  Rays 

An  optical  ray   (X°;X)  for  the  maximization  problem  (1.7) 
is  a  set  of  optimal  vectors  of  the  form  X°  +  \X,      where   X 
takes  >n  all  nonnegative  values,   X°  is  a  fixed  optimal  vector, 
and  X  is  any  fixed  vector  which  has  been  normalized  so  that  the 
sum  of  its  components  is  one.   Optimal  rays  for  the  minimization 
problem  are  defined  analogously.   The  vectors,   X,   are  called 
directions  of  the  optimal  rays  and  the  following  Lemma  character- 
izes them. 

Lemma  6.1.   Suppose  the  sum  of  the  components  of  the  vector 
X   is  one,  and  that  X   is  an  optimal  vector  for  the  maximization 
problem  (1.7).   Then,   (X°,X)   is  an  optimal  ray  if,  and  only  if, 

(1)  XfcO 

(2)  AX^O 

(3)  CX=0. 

The  dual  statement  holds  as  well;   (2)  becomes  UA5  0. 

Proof:   The  set  of  optimal  vectors   (X°;X)   is  an  optimal  ray  if, 
and  only  if, 

(i)   AX°  +  XAX  SB 
(ii)   CX°  +  XCX=CX° 
(iii)   X°  +  AXkO   for  all  XfcO. 

Clearly,  these  conditions  are  satisfied  if,  and  only  if,  (1), 
(2),  and  (3)  hold. 
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Ler,.;;ia  6.2.   The  set  of  directions  of  optimal  rays  of  a 
linear-programming  problem  has  a  finite  number  of  extreme  vectors 

:Jroof:   According  to  Lemma  6.1»   X  0  is  the  direction  of  an 
optimal  ray  for  the  maximization  problem  if,  and  only  if, 

(i)   Ex.^1 

di)    E(-x.)£i 

(iii)   AXiO 
(iv)   CX=0 
(v)   (-CX)iO. 
Relations  (i)  through  (v)  can  be  considered  the  row  constraints 
of  a  suitable  new  maximization  problem.   The  desired  conclusion 
follows  by  applying  Corollary  3.1  to  this  problem,   of  course, 
an  analogous  argument  holds  for  the  minimization  problem. 

The  discussion  of  the  theory  necessary  to  prove  the  follow- 
in,:;;  theorems  is  quite  lengthy  and  involves  concepts  which 
have  not  been  considered  in-  this  report.   For  this  reason, 
these  theorems  will  be  stat<  d  without  proof.   Theorem  6.1  is 
needed  to  prove  Theorem  6.2,  a  statement  which  characterizes 
trie  set  of  all  optimal  vectors  of  a  linear-programming. problem. 
A  proof  of  Theorem  6.1  can  be  found  in  (7)  and  a  proof  of 
Theorem^  6.2  can  be  found  in  (3). 

Theorem  6.1.   Let  3=  (x  |  AX  ^  Bj  be  a  nonvacuous  set  with 
the  matrix  A  having  linearly  independent  columns.   Then  3 
has  a  minimal  basis   (?1 , . . . ,?  •Q1, . . . ,Q  )   which  is  unique 
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(up  to  positive  multiples  of  the  Q.'s).  Here,  (P,  ,...,1"  )  is 
the  set  of  extreme  vectors  of  S,  and  (Q,,... ,Q  )  is  the  set 
of  directions  of  the  extreme  vectors  of  S. 


:me 


Theorem  6.2.   Let  |Xrj-   be- the  finite  set  of  extrei 
optimal  vectors  for  the  maximization  problem  and  {xs}  be  the 
finite  set  of  extreme  directions  of  optimal  rays  for  the  maximi- 
zation problem.   Then,  the  set  of  all  optimal  vectors  X  is 
the  set  of  all  vectors  of  the  form  X=H~\Xr  +H/U  Xs   with 

■i-  o 

all  Xr^0,      all  /^iO,  and  ZT^-1.   Of  course,  the  dual  state- 
ment holds  for  the  minimization  problem. 
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Lagrange  Multipliers 

The  traditional  calculus  procedure  for  finding  a  constrained 
maximum  or  minimum  employs  Lagrange  Multipliers.   This  procedure 
may  be  adapted  to  linear  programming  problems  to  derive  a  necessary 
and  sufficient  condition  for  the  vectors  X   and  U   to  be  optimal 
vectors  for  the  dual  programs  (1.7)  and  (1.8). 

The  general  procedure  of  Lagrange's  method  will  now  be  sum- 
marized.  If  a  given  function  G(x-,,...,x  )=G(X)   is  to  be  maxi- 
mized or  minimized  subject  to  the  constraints,  ?-,  (X)=0, . . .  ,F  (X)=G, 
then  the  first  step  is  to  form  the  Lagrangian  function 
K(X,u, , . .. ,ii  )=H(X,U)=G(X).  +  £_u.F.(X).   Then,  the  necessary 
conditions  that  H(X,U)   have  an  unconstrained  extremum  are  that 
the  n  first  partial  derivations  of  H(X,U)   vanish.   These  con- 
ditions are  also  necessary  for  G(X,U)   to  have  a  constrained 
extremum,  so  that  the  free  extreme  values  of  K(X,U)   are  sought, 
among  which  will  be  the  extreme  values  of  G(X,U). 

A  similar  situation  arises  in  linear  programming,  where  the 
object  is  to  eoetremize  a  linear,  function  subject  to  constraints 
which  are  linear  inequalities,  rather  than  equations.   To  develop 
a  technique  analogous  to  Lagrange's  method,  the  Lagrangian  func- 
tion 

(7.1)     L(X,U)  =  CX  +  UB  -  UAX 

■ cx  +  f  ui(bi  -pW 

is  formed.   The  second  form  of  L(X,U)   in  (7.1)   exhibits  a 
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sibuation  analogous  to  that  described  above.   The  problem  described 
here  is  a  maximization  problem  with  G(X)  =  CX,   F.(X)  =  (b.  -V  a.  .x.). 
and  the  u^   regarded  as  multipliers.   Similarly,  the  third  form 
of  L(X,U)  in   (?.l)  exhibit;:  the  analogy  for  the  minimization 
problem.   The  following  theoi^em  justifies  this  analogy. 

Theorem  7.1-   A  necessary  and  sufficient  condition  that  X°^0 

and  U  =0  be  optimal"  vectors  for  the  dual  programs  (1.7)  and  (1.8) 

is  that   (X°,U°)   be  a  "saddle  point"   for  L(X,U)   in  the  sense 
that 

L(X,U0)^L(X°,U0)iL(X°,U) 

for  all  X^O  and  U^O.   If  .  X°   and  U°  are  optimal  vectors, 
then  L(X  , U  )   is  the  optimal  value  of  the  dual  programs. 

Proof:   To  prove  the  necessity  of  the  condition,  suppose  that 
X   and  U°  are  optimal  vectors.   By  Corollary  3.1,   CX°=U°B. 
Since  CX0^  U°AX°^  U°3,   it  follows  from  (7.1)  that 
L(X°,U°)  =  CX°  =  U°B.   Thus,  for  any  XSO,   U^O, 

L(X,U°)  =U°B  +  (C  -  U°A)XtU0B=L(X°,IJ0)   and 
L(X°,U)  =CX°  +  U(B  -  AX0)*  CX°=L(X°,U0). 

Therefore,  the.  point   (X°,U°)   is  a  saddle  point. 

To  prove  the  sufficiency  of  the  condition,  suppose  that 
(X  ,U  )   is  a  saddle  point  with  X0^  0  and  U°S0.   Then, 
for  any  XlO, 

U°3  -  (U°A  -  C)X  =  L(X,U°)1L(X°,U°)  -U°B  -  (U°A  -  C)X° 
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or 


(7.2)      (U°A  -  C)(X  -  X°)  *0. 

Now,  since  X  is  arbitrary,  choose  X  =  X°  +  F . ,   where  S. 

is  the  j-th  unit  vector.   The::,  (7.2)  yields   (U°A  -  C),.£  0. 

J 

Let   j  =  l,...,n,   and  it  follows  that  U°A£C.   Hence,   U° 
is  feasible.   Similarly,  for  any  vector  USO, 

CX°  +  U°(B  -  AX°)=L(X°,U°)1=L(X0,U)=CX0  +  U(3  -,  AX°) 


or 


Ot(U  -  U°)(B  -  AX°). 

Therefore,  choose  U  =  U°  +  E^'j   let  i-l,...,m,   and  it  follows, 
as  above,  that  B£AX°.   Hence,   X°   is  feasible. 

Since  L(0,U°)tL(Xo,U°)  =  L(Xo,0),  U°B*CX°.   By  Lemma  1.1, 
CX  |  U  B.  .Hence,   U°B=CX°  and  by  Lemma  1.2,  vectors  X°   and 
U°  are  optimal.   Therefore,   U°B=L(X0,U0)=CX°. 

It  is  to  be  noted  that  the  situation  involving  dual  linear 
programs  to  which  the  method  of  Lagrange  has  been  applied  hei-e 
differs  from  the  ordinary  situation  in  the  calculus.   First,  the 
fact  that  a  maximization  problem  and  a  minimization  problem  are 
being  dealt  with  simultaneously  leads  to  the  consideration  of  a 
saddle  point  of  L(X,U).   Second,  the  conditions  involved  are  both 
necessary  and  sufficient.   Finally,  the  Lagrangian  function 
L(X,U)   is  subject  to  the  constraints  X^O  and   U^O. 
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Appendix 

One  of  the  best  known  and  the  most  widely  used  computational 
methods  of  solving  a  linear  programming  problem  is  the  Simplex 
Method  developed  by  G.  B.  Dantzig  {'■}).      The  purpose  of  this 
section  is  to  discuss  this  method. 

The  linear-programming  problem  to  be  discussed  in  this 
section  will  considered  in  the  following  standard  form: 

Find  the  values  A.  ,  .  .  .  ,  A   which  maximize  the  linear 
function 


(1) 


z  =  A  c     + .. ..  +  A  c 

oil        n  n 


subject  to  the  conditions  that 


(2) 
and 
(5) 


A.,o 


Vn  +-"  +  Vm=bi' 


A,  a  t  +  •  •  -  +  A  a  =  b  , 

1  ml        n  mn  m' 


( 3-1, ...,n) 


.   are  constants   (i-l,..,,m;  j=l,...,n). 


where   a .  . ,   b . ,   c 

The  general  linear-programming  problem  may  involve  con- 
straints which  are  inequalities  or  a  mixture  of  equations  and 
inequalities,  ana  variables  which  can  have  negative  values, 
and  a  linear  function  which  is  to  be  minimized  rather  than 
maximized,   ouch  problems  are  easily  transformed  to  the  standard 
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form  stated  above. 

If  a  constraint  is  a  less-than  condition,  so  that 
ailxl  +  '*,+  ainxn=^i'   then  tne  constraint  may  be  transformed 
into  an  equation  by  adding  a  nonnegative  slack  variable  s. 

to  the  left-hand  side  and  writing  a . , x,  +  •••  +  a,  x  +  s.=b.. 

°   il  1        In  n    11 

The  slack  variable   s,   is  an  additional  unknown  that  has  to  be 
determined.   Similarly,  if  a  constraint  is  a  greater-than  condi- 
tion, so  that  a^  +•••+  ainxn-bi>   then  one  can  write 
ailxl  +,,'f  ainxn  ~  sja^i»   where   s,   is  again  a  nonnegative 
slack  variable.   By  introducing  a  nonnegative  slack  variable 
for  each  constraint  that  is  an  inequality,  one  can  always  express 
the  constraints  in  the  standard  form  of  (3). 

If  some  variables  are  not  constrained  to  be  nonnegative, 
then  those  variables  can  always  be  expressed  as  the  difference 
of  two  nonnegative  variables.   Substituting  the  difference  of 
two  nonnegative  variables  into  (3)  for  those  variables  which  are 
not  restricted  in  sign  will  result  in  a  problem  in  standard  form. 

Finally,. if  one  wishes  to  minimize  the  function  (1),  rather 

than  maximize  it,  one  may  reverse  the.  signs  of  all  the  c.   in 

J 
(1)   and  then  proceed  to  maximize  the  resulting  function. 

iSach  column  of  coefficients  in  (3)  may  be  viewed  as  represent- 
ing the  components  of  a  vector  in  the  vector  space   V  .   Let  P 

m         J 

denote  the  j-th  column  of  coefficients  and  P   the  column  of 


c 


constants  in  (3).   Then,  by  definition, 


w 
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(?!,..., Pn;  p0) 


a-,  t  a,  -•  •  •  a,  b, 
11  12     In  1 


a  ,  a  0  *  •  •  a  b 

Eil  m2     mn  m 


The  basic  problem  then  is  to  determine  nonegative    \  %   0 

such  that 


(5) 


(6) 


Vl  + 


\cl 


+    X  p  =p  , 

n  n  o' 


+  A c   =z=max. 


A  set  of   Ai  which  satisfy  (.;)  without  necessarily 
yielding  the  maximum  in   (6)  will  be  termed  a  feasible  solu- 
tion; one  which  maximizes  (6). will  be  called  a  maximum  feasible 
solution.   Having  defined  the  problem,  the  simplex  method  for 
finding  its  solutions  may  now  be  discussed.   The  simplex  tech- 
nique consists  of  constructing  first  a'* feasible,  and  then  a 
maximum  feasible,  solution.   Since  feasible  solutions  are 
frequently  obtained  by  inspection' and .because  an  arbitrary  feasible 
solution  can  be  obtained  in  a  manner  analogous  to  the  construc- 
tion of  a  maximum  feasible  solution,  the  construction  of  a  maxi- 
mum, feasible  solution  from  a  given  feasible  solution  will  be  con- 
sidered first. 

To  simplify  the  development  that  follows,  the  following 
nondegeneracy  assumption  will  be  made:   Every  subset  of  m  vec- 
tors  from  the  set   (P^Pp. . .  ,Fn)  is  linearly  independent. 
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However,  modifications  to  the  method  have  been  developed  that  will 
solve  any  problem  they  may  arise  which  is  degenerate. 
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Construction  of  a  Maximum  Feasible  Solution 

Assume  as  given  a  feasible  solution  consisting  of  exactly 
m  of  the  "X.      nonzero,  that  is, 

(7)      A,P,  w  +  ajp-p'     X>0. 
11         m  m  o '      i 

(Q)        ^-ict  +  •••  +  Xc  =z  . 
ll         m  m  o 

The  set  of  m  vectors,   P.,   in  (7)  will  be  referred  to  as  a 

J 

hasis  J'or  Pl»,,,'Pn*   In  establishing  the  conditions  for  the 
construction  of  a  maximum  feasible  solution,  it  will  be  necessary 
to  express  all  n  vectors,   P.,   in  terms  of  a  basis,  that  is, 

(9)  x,  .P,  +..-+  x  .P  -P.  •       (i=l,...  n). 

lj  1        mj  m  j  v»i*»  *  ••  ».**/• 

Now,  define   z .   by 

(10)  x,  .c,  +•••+  x  .c  =z.  (i=l     n) 

lj  1        mj  m  j  v j  x, . . . taj . 

Theorem  1.   If,  for  any  fixed  j,  the  condition 

(11)  c.>z. 

J    J 

holds,  then  a  set  of  feasible  solutions  can  be  constructed 
such  that 

(12)  z>zQ 

for  any  member  of  the  set,  where  the  upper  bound  of  z  is  either 
finite  or  infinite. 

Case  I:   If  finite,  a  feasible  solution  consisting  of 
exactly  m  ^>  0  can  be  constructed. 
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Case  II:   If  infinite,  u  class  of  feasible  solutions  con- 
sisting of  a  set  of  exactly  (m* +1)   positive   A.   can  be  con- 
structed such  that  the  upper1  bound  of  z=  +  oo  . 

Proof:   Multiplying  equation  (9)  by  9  and  substracting  from 
equation  (7),  and  multiplying  (10)  by  9  and  substracting  from 
(8),  the  equations 

(13)      (\-««y)Pi  — •♦  (\-e\,d)Pm  +  9P0=Po' 

(»)  (  \  -  ityioj    t->*    (  7l„,  -  exmJ)om  ♦  eod 

=zo  ♦  e(cj  -  Zj), 

are  obtained  where  the  term  9c.   has  been  added  to  both  sides 

J 

of  (14). 

Since  a^  0  for  every  i   in  (13),  it  is  clear  that  there 
is,  for  92  0,  either  a  finite  range  of  values   9  >  0^0  or 
an  infinite  range  of  values  such  that  the  coefficients  of  the 
Pi  remain  positive.   It  follows  from  (14)  that  the  function   z 
of  this  set  of  feasible  solutions  is  a  strictly  monotonlcally 
increasing  function  of  9, 

(15)  z=z  +  9(c.  -  z.)>z        '9>0, 

sinc6   c  ">  z 

J    J   by  hypothesis   (11).   This  establishes  (12). 

Case  I:   If  xi->  0  for  at  least  one   i=l,...,m  in  (9) 

or  (13),  the  largest  value  of  9  for  which  all  coefficients 

in  (12)  remain  nonnegative  is  given  by 

(16)  9  =min(7L./x,  .),    x.  .>0. 
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If  for  some  value  i   of  i,   0  in  (16)  exists,  it  is  clear 
that  the  coefficient  corresponding  to   i   in  (13)  and  (14-)  will 
vanish.   Hence,  a  feasible  solution,  given  by  ©=9  ,   has  been 
constructed  with  exactly  m  positive  weights  "X.  .   Further- 
more,  z>zQ.   It  should  be  noted  that  this  new  set  of  m  vectors 
consists  of  the  new  vector,   P.,   and  (m  -  1)   of  the  original  m 
vectors.   This  is  a  desired  solution  for  Case  I. 

The  new  set  of  m  vectors  may  now  be  used  as  a  new  basis, 
and  again,  as  in  (9)  and  (10),  all  vectors  may  be  expressed  in 

terms  of  the  new  basis  and  the  values  of  the   c^   compared  with 

<j 

the  values  of  the   z.   just  computed.   If  any  c>  z.,   the  value 
of  z  can  be  increased.   If  at  least  one  x. .  >  0,   another  new 
basis  can  be  formed.   Now,  assume  that  this  process  is  continued 
until  it  is  not  possible  to  form  a  new  basis.   This  must  occur  in 

a  finite  number  of  steps  because  there  are  at  most   C   bases 

n  m 

and  none  of  these  can  recur,  for  in  that  ca.se  the  value  of  z 

would  also  recur.   This  cannot  happen  because  the  function  z 

is  strictly  increasing.   Thus,  it  is  evident  that  the  process  must 
terminate,  either  because  at  some  stage 

C1?)       Xij~°  f0P  a11   i  =  1>--->m   and  some  fixed   j, 
or  because 

(13)       c  <  z.   for  all   j=l,...,n. 

Case  II:   If  inequality  (17)  occurs,  then  it  -is  clear  that 
6  has  no  finite  upper  bound  and  that  a  class  of  feasible 
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solutions  has  been  constructed  consisting  of  a  set  of  (m  +  1) 

positive   A-   such  that  the  upper  bound  of   z=+oo. 

In  all  problems  in  which  there  is  a  finite  upper  bound  to 

z,   the  iterative  process  must  necessarily  lead  to  the  condition 

(18).   However,  it  will  be  proved  in  the  next  theorem  that  the 

feasible  solution  associated  with  the  final  basis,  which  has 

the  property  that   c.iz,   for  all   j  =  l,...,m   is  also  a  maximum 

J    J 

feasible  solution.   Hence,  in  all  problems  in  which  there  is  no 
finite  upper  bound  to   z,   the  iterative  process  must  necessarily 
lead  to  the  condition  (17).   Furthermore,  by  rewriting  equation 
(9)  as 

for  the  fixed  j  of  (17),  a  nonnegative  linear  combination  of 

(m  +  1)  vectors  has  been  shown  to  vanish  if  the  upper  bound  of 

z   is  +co . 

As  a  practical  computing  matter,  the  procedure  of  progressing 

from  one  basis  to  the  next  does  not  involve  as  much  computation 

as  would  first  appear,  because  the  basis,  except  for  the  deletion 

and  insertion  of  one  vector,  is  the  same  as  before.   In  fact, 

one  iteration  involves  less  than  mn  multiplications  and  an 

equal  number  of  additions.   It  has  been  observed  from  experience 

that  the  number  of  iterations  can  be  greatly  reduced  not  by 

arbitrarily  selecting  any  vector,   P,,   satisfying  c.>z.,  but 

J  <3    J 

by  selecting  the  one  which  ,:ives  the  greatest  immediate  increase 
to  the  function  z. .  From  (15),  the  criterion  for  the  choice  of 
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j   is  such  that 

e  (c.  -  z.) 
o  j   y 

is  a  maximum,  where 

9   is  given 

by  (16), 

A  criterion  that  involves  considerably  le 

ss  computa- 

tion  and  seems  to 

yield  just  a 

s  satisfactory  result 

s  is  to 

choose 

j   such  t 

hat   (c.  -.a. 

J    J 

)   is  a  maximum.   In 

either  case, 

appro;-:  ii 

cately  m 

changes  in  b 

asis  are  encountered 

in  px^actice, 

so  that 

2 
about  m 

n  multiplies 

tions  are  involved  in 

obtaining  a 

maximum 

feasible 

solution  frorr 

i  a  feasible  solution. 

Theorem  2. 

If,  for  all 

j=l, . . . ,n,   the  condi 

tion 

holds,  th 

en  equations. 

(?)  and  (3)  constitut 

e  a  maximum 

feasible  solution 

• 

Proof: 

Let 

(19) 

MlPl  + 

•••  +  *nPn=Po. 

M^O, 

(20) 

/W1c1  + 

* 

>  "  +  M   c  =z  , 

n  n   ' 

constitu 

ite  any  other  feasible 

solution.   It  will  be 

shown  that 

zo=z+- 

By 

hypothesis,   c  .  1=  z  . , 
J    J 

so  that  replacing  c  . 

by  z .   in 

(20)  gives 

(21) 

A1z1  + 

+  ^nzn^z* 

• 

On 

substitut. 

Lng  the  value 

of  P.  given  by  equa- 

tion  (9)  into 

equation 

(19)  and 

the  value  of 

z.   given  by  (10)  into  (21), 

it  folio 

ws  that 

(22) 

(  th^h  *''•'; 

* ( £  ^vvp0> 
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(23) 


c  ^/vu)ci  +--,+  (  hMix-i)c--z  • 


According  to  the  assumption  of  nondegeneracy,  the  corresponding 

coefficients  of  P.   in  equations  (7)  and  (22)  must  be  equal. 

Hence,  the  ine-ualitv  (23)  becomes   A,  c,   +.»«+   A  c  5.  z 

11  m  m  _    T 


or ,  by  O ) ,   z  >  z  . 


In  order  that  another  maximum  feasible  solution  exist,  it 

;cessary  that   c  .= 
J 

However,  in  this  case, 


is  necessary  that   c.=z.   for  some   P.   not  in  the  final  basis. 


(24) 


Pl  P2  ?n< 


cl  c2   cn 


has  at  least  one  set  of   (m  +  1)  columns  which  are  linearly 
dependent.   Thus,  a  sufficient  condition  that  the  maximum  fea- 
sible solution  constructed  from  the  given  feasible  solution  be 
unique  is  that  ever  set  of   (m  +  1)   vectors  defined  by  columns  in 
(24),  be  linearly  Independent. 

Construction  of  a  Feasible  Solution 

Consider  the   (n  +  1)  column  vectors  in  (4).  '  Select  an 

arbitrary  basis  of   (m  -  1)   vectors,   P.,   and  the  vector  t> 

d  o 

from  this  set.   Denote  the  basis  by   (PQ ;P1, . . . ,p   ) .  .Now,  any 
P.   can  be  expressed  in  terms  of  this  basis  by 

(2/°       yo/o  +  ^l/l  -•-  y<»-l)jW*j,    <j  =  l,...,n). 

Theorem  3.   A  sufficient  condition  that  there  exist  no 
feasible  solution  is  that  y  ,  £0  for  all   j. 
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Proof:   Assume  that  there  does  exist  a  feasible  solution, 

(25)       Vl  +-"+  Vn-Po-     V0' 

On  substituting  the  expressions  for  P.   given  by  (24-)   into 
equation  (2p) ,  it  follows  that 


Vi<  £  v(p.i)j)-o. 


By  virtue  of  the  assumed  independence  of  (P  ;?•,,..., P  _-,  ) , 
it  is  evident  that  each  coefficient  in  (26)  must  vanish.   In 
particular,    z_  A.y.  .  -  1=0.   This  is  impossible  if  both 
LlO  and  yni^0   for  all   j. 

To  construct  a  feasible  solution,  a  fixed  reference  vector 

R  is  first  defined.   R  is  given  by  R=w,  P,  +  ••«  +  w   ,P   ,-kP, 

°      "  11        m-1  m-1    o  o' 

where  wi>  0   (i=l, . . . ,m-l)   and  kQ>  0  are  arbitrarily  chosen. 
The  above  equation  may  be  rewritten  in  the  form 

(27)       R  +  k  P  =w  p  ♦.•*+  wm  ,P.  .. 

o  o  li        m-1  m-1 

In  the  development  that  follows,   k   will  play  a  role  analogous 

to   2Q. 

By  Theorem  3,  if  there  exists  a  feasible  solution,  there 
exists  at  least  one   j   (which  will  be  considered  to  fixed)  such 
that  yQ^>  0.   Multiplying  (24-)  by  9  and  substracting  from  (27), 
it  follows  that 
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For  a  rarve  of  0  >0>O,   one  can  construct,  in  a  manner 
■        o      '  ' 

similar  to  (13)  and  (14),  a  set  of  vectors  of  the  form  R  +  kP  , 

where  each  is  a  positive  linear  combination  of  the  vectors  P.. 

J 

Since   k  will  play  a  role  analogous  to   z,   the  problem  is  to 

find  the  largest  value  k   Cor  which  this  construction  is  possible. 

It  should  be  noted  that  k=k  +  ©v  .  >  k   since  y  .  >0  has 

o    "OJ    o         ^oj 

been  assumed. 

If,  in  the  equation  (24),  all  y.-^O  (i  =  l, .  . .  ,m-l)  ,   the 

*  J 

coefficients  of  P.   will  be  positive  and  k -*+ co  as  0  — s>  +  co  . 

J 

Also,  it  will  be  seen,  by  solving  (24)  for  P  ,   that 

Po=(l/VPj  +  (-ylj)Pl   +-"+  (-y(m-l)A>a)Pm-l> 

and  that  a  feasible  solution  has  been  obtained.   That  is,   P 

'    o 

has  been  expressed  as  a  positive  linear  combination  of  P, ,...,P  , 

l '    '  m-1 

and  P..   If  at  least  one  y, .>  0   (i=l , . . . ,m-l) ,   the  largest 
value  of  0  is  then 

(2d)       e0=min(w./yiJ),      y...>  0. 

When  0   is  set  equal  to   0  ,   the  coefficient  of  at  least  one 
vector,   P^,   will  vanish  and  a  new  vector  R  +   k,P  ,   will  be 
formed  from  (27)  which  is  expressed  as 'a  positive  linear  combi- 
nation' of  just   (m  -  1)   vectors,   P.,   where   k,  =k  +  0  y  .>k  . 

1  1   o     o*7  OJ    o 

By  expressing  all  vectors  P.   in  terms  of  the  new  basis, 
the  process  may  be  repeated,  each  time  obtaining  a  larger  value 
of  k,  i.e.,  a  feasible  solution.   The  process  must  terminate 
in  a  finite  number  of  steps.   Otherwise,  since  there  is  only  a 
finite  number  of  b-:ses,  the  same  combination  of   (m-1)   points 
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P.   would  appear  twice.   That  is, 

(29)       R  +  k'P  »w'P,  +.-.+  W'   P  , . 

oil        m-1  m-1* 


(50) 


R  +  k"P  =w''P,  +...+  w"  ,P  lf 

o   11        m-1  m-1' 


where  k">k'.  Subtracting  equation  (29)  from  equation  (JO),  a 
nonvanishing  expression  giving  P0   in  terms  of  (m-1)   vectors 
i^,   is  obtained.   This  contradicts  the  nondegeneracy  assumption. 
There  are  only  two  conditions  which  will  terminate  the  pro- 
cess.  That  is,  after  a  finite  number  of  iterations,  either 
y  •  £  0  for  all   j  =  l,...,n,   or,  for  some  fixed  j,   y.  . £  0 
for  all   i=l,...,m.   In  the  first  case,   Theorem  5  implies  that 
no  feasible  solution  exists.   In  the  latter  case,  -equation  (?4) 
may  be  solved  for  PQ,   as  was  done  in  (28),    to  obtain  the  desired 
feasible  solution. 
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Linear  programming  is  usually  thought  of  as  referring  to 
techniques  for  solving  a  general  class  of  optimization  problems 
dealing  with  the  interaction  of  many  variables ' subject  to  cer- 
tain linear  restraining  conditions.   In  solving  these  problems, 
certain  objectives  are  to  be  obtained  in  the  best  possible,  or 
optimal,  fashion  subject  to  certain  restraining  conditions  which 
may  arise  from  a  variety  of  sources.   However,  the  phases  of 
linear  programming  discussed  in  this  report  are  primarily  exist- 
ence discussions.   The  linear-programming  problem  is  first  de- 
fiaed  in  terms  of  a  system  of  linear  inequalities,  and  then, 
certain  conditions  necessary  for  the  existence  of  a  solution  are 
derived. 

The  concept  of  duality  is  employed  whenever  possible.   If 
the  linear-programming  problem  is  stated  in  terms  of  seeking  the 
maximum  value  of  a  linear  function  subject  to  a  system  of  linear 
constraints,  then  the  dual  to  this  problem  is  the  search  for  a 
minimum  value  for  an  associated  linear  function  subject  to  an 
associated  system  of  linear  constraints. 

As  background  necessary  to  prove  the  fundamental  Duality 
and  Existence  Theorems,  dual  systems  of  homogeneous  linear  rela- 
tions are  first  discussed.   These  theorems  give  necessary  and 
sufficient  conditions  for  the  existence  of  feasible  and  optimal 
solutions  for  the  dual  linear-programming  problems. 

Dual  linear  programs  involving  systems  of  mixed  constraints 
are  shown  to  be  equivalent  to  the  dual  linear  programs  as  origi- 
nally defined. 


From  a  study  of  linear  programs  involving  systems  of  equated 
constraints,  the  set  of  all  optimal  solutions  is  character-ized. 

A  modified  version  of  the  method  of  Lagrange  Multipliers  is 
used  to  derive  a  necessary  and  sufficient  condition  for  the  exist- 
ence of  optimal  solutions  for  the  dual  linear  programs. 

In  the  appendix,  the  so-called  simplex  method,  developed 
by  George  B.  Dantzig,  is  discussed.   It  was  the  inability  to  find 
an  analytic  solution  to  the  linear-programming  problem  that  led 
to  the  development  of  the  simplex  method  and  various  other  itera- 
tive methods.   The  simplex  method  is  the  most  widely  used  method, 
since  experience  has  shown  it  to  be  a  rather  efficient  method 
from  a  computational  point  of  view. 

While  it  is  true  that  the  theory  of  convex  polyhedral  cones 
is  but  a  geometric  interpretation  of  the  theory  of  linear  inequal- 
ities, this  theory  is  quite  extensive,  and  as  a  result  is  not 
emphasized  in  this  report.   Therefore,  no  attempt  is  made  at 
geometrical  interpretation  of  any  concept.   In  fact,  all  results 
are  stated  and  proved  algebraically,  as  properties  of  linear  in- 
equalities. 


